Collection #1: 1.4 Billion Password Breach

Posted by & filed under Penetration Testing.

Collection #1

Collection #1 is almost two times larger than the previous largest credential exposure.

This is not just a list. It is an aggregated, interactive database that allows for fast (one second response) searches and new breach imports. Given the fact that people reuse passwords across their email, social media, e-commerce, banking and work accounts, hackers can automate account hijacking or account takeover.

The dump includes a file called “imported.log” with 256 corpuses listed, including and with added data from all those in the Exploit.in and Anti Public dumps as well as 133 addition or new breaches.

Structure

The data is structured in an alphabetic directory tree fragmented in 1,981 pieces to allow fast searches.

Freshness

Although the majority of the Collection #1 breaches are known within the Breach and Hacker community, 14% of exposed username/passwords pairs had not previously been decrypted by the community and are now available in clear text.

This new breach adds 385 million new credential pairs, 318 million unique users, and 147 million passwords pertaining to those previous dumps.

Top Passwords

Original Source

query.sh

Included in the source is “query.sh”. You can execute this to do a lookup for passwords on a particular email address.

Generate Wordlist

Word Count

breachcompilation.txt ==> 1 012 024 699 breachcompilation.txt

Sort

Remove Trailing Spaces

Recount

breachcompilation.sorted.txt ==> 384 153 427 breachcompilation.sorted.txt

Download

breachcompilation.txt (8.7G)

Conclusion

This experience of searching and finding passwords within the collect #1 database is as scary as it is shocking. The best ways to get around this is use a password manager and create complex 12+ character long strings and rotate (at least your critical credentials) regularly.

Ansible – Build A Mail Server Using Postfix, Dovecot, MySQL, SpamAssassin And More…

Posted by & filed under Ansible, Postfix.

Overview

If you need to build your own Postfix, Dovecot, SpamAssassin and Roundcube server for your own mail hosting, look no further than this all in one Ansible script. Just configure the global configuration file, run the script and BAM, you have a fully functioning mail server! This script completes an end to end configuration of the server and covers the following:

  1. Apache
  2. PHP 7.2
  3. MySQL
  4. Postfix
  5. Dovecot
  6. Letsencrypt
  7. Sieve
  8. SpamAssassin
  9. Postgrey
  10. iptables
  11. Roundcube Webmail with managesieve & two_factor plugins

Test Bed

  • Ansible control server running Ubuntu 18.04 LTS
  • Test server running Ubuntu 18.04 LTS

Requirements

  1. Ansible control server
  2. SSH keys established between Ansible control server and destination server(s)
  3. 2x Public DNS A records pointing to the server to be set up

Role Dependancies

I use two Ansible Galaxy roles, one to setup iptables and one for Letsencrypt SSL certs (I was just too lazy to code that up all myself) You will need to run the following commands to download the Ansible Galaxy roles onto the Ansible control server:

Git Clone

I have all the yaml and conf scripts sitting in a public Github repository which can be cloned by running the following:

Conclusion

This will set up a complete mail server based on Digital Oceans how to and is expanded to include a few more services. I haven’t broken this script into roles yet and is in a “MVP” form for now. Questions and comments are always welcome as usual.

Automate SSH Key Rotation on Ubuntu with Ansible

Posted by & filed under Ansible, Hardening, Linux & Unix.

Overview

Changing your SSH keys is as important as changing your underpants daily, running this script on a frequent basis will ensure access to the servers are changed on a regular basis. Use Ansible to do ssh key rotation in your sleep!

Test Bed

  • Ansible control server running Ubuntu 18.04 LTS
  • Test server running Ubuntu 18.04 LTS

Requirements

  1. Ansible control server
  2. SSH keys established between Ansible control server and destination server(s)
  3. A folder called “pubkeys” where the script is running from

Break Down

  1. Creates a new directory on the remote server to generate the new keys on
  2. Generates the new key pair in the newly formed folder
  3. Copies the new public key to the local machine running the ansible script under /pubkeys/ and names it “id_rsa.%hostname%.pub
  4. Removes existing private key
  5. Removes existing public key
  6. Moves new private key to the users .ssh folder
  7. Moves new public key to the users .ssh folder
  8. Changes new private key to read only
  9. Invalidates existing keys and applies the public key copied to the local host to the server
  10. Copies the new private key local host and changes the file to “id_rsa.%hostname%
  11. Removes “newsshkey” folder on remote host as a clean up

ssh_key_rotation.yml

Note: You will need to change/remove the “- hosts:” entry

Key Management

To fully automate this I have mounted a cifs share and created a symbolic link on the Ansible server from the ~/.ssh folder to the cifs share. All my other clients are set up the same way so when you update the key it copies the key to a central repository which all other clients are symbolically linked to.

Conclusion

This can be greatly be improved on but is a good starting point in the rotation of your ssh keys. I’m happy to hear suggestions on how this could be improved.

Ansible – mySQL root password change on Ubuntu

Posted by & filed under Ansible, Linux & Unix, mySQL.

 

This Ansible script will fully rotate your MySQL root account passwords (or change any MySQL account passwords if you change the script) and implement my.cf so you don’t have to keep putting the password in. This took me a while to figure out, there are 

Test Bed

  1. Ansible control server running Ubuntu 18.04
  2. Ubuntu 18.04 Bionic test server running mySQL 5.7.25

Requirements

  1. Ansible control server
  2. SSH keys established between Ansible control server and destination server

Overview

  1. Install mySQL package with required dependancies
  2. Stop mySQL service
  3. Set mySQL environment variables
  4. Start mySQL
  5. Change mySQL root password to a mySQL native password (native is very important!)
  6. Copy .my.cnf from local source to ~
  7. mySQL flush privileges
  8. Stop mySQL service
  9. Unset mySQL environment variables
  10. Start mySQL

sql.yml

etc/mysql/.my.cf

global-vars/config.yml

Execution

Other Considerations

You will need to remove lines 7 to 19 if you are not installing MySQL for the first time.

If any applications are using the account you are rotating, the application will auth fail (I would hope your not using root for app authentication) – if you use this against any other username this will need to be considered.

Conclusion

The gotcha for a lot of people (from what I’ve read on blogs/github) is that when the mysql root password changes you also need to change it from “auth_socket” to “mysql_native_password”. 

 

Regenerating SID’s using Powershell

Posted by & filed under Microsoft, Powershell.

I’ve devised a handy way of regenerating SID’s on Microsoft operating systems using a third party tool and a little PowerShell magic.

The Breakdown

The first part of the script sets variables that can be called upon to randomise the SID number.

The next part sets the source location to download newsid.exe and defines a destination location.

The “set-itemproperty” adds a key into the registry to trick Newsid to think it has already accepted the EULA.

The “start-bitstransfer” initiates the download of newsid.

The real magic is in the last part – newsid is executed with “/a” to run with no prompts and “/n” to not reboot after.

execution”. The “S-1-5-21-“ is the first part of any SID then it calls the three random numbers generated from $random%.

The Script

Considerations

Having this script on a gold imaged server set to auto run on login (with a restart afterwards) would be the best implementation of this. You wouldn’t need to use the source and destination either and just point to a local copy of newsid.exe. I did however write this with a remote copy of newsid.exe in mind.

You will need to change the source and destination to suit your needs.

This is written for Microsoft PowerShell Version 5+.

You could drop the “/n” in the newsid.exe switch so it does reboot on completion.

The “S-1-5-21-” at the beginning of the SID is always going to be the same.

Conclusion

With the ever increasing demand for automating the deployment of servers, this is as important as ever if you start interconnecting the automated servers.