File Integrity Monitoring systems are great for notifying users when important files are being changed, and can even prevent the changes from sticking. In this post, I’ll show you how to build your own basic FIM using Python, alerting on changes by sending messages to the console.
Note: I’ve created a new directory for this project, and added a few junk files in addition to my
BasicFIM.py script. You can add any files you want.
##Getting the files to monitor The first step we’ll take, is to gather up all the files in our script’s directory. After all, we need something to monitor, right?
- Line 1: We start by adding
import osto the top of our script, giving us access to Python’s built-in OS module. We’ll need this for tasks like grabbing directories and determining their contents.
- Line 3: This line has a lot going on, so I’ll break it down into pieces
for item in os.listdir('.'): The
listdir()function is used to retrieve all items found in a given path. Here, we use
'.', which is the string constant for the current working directory.
if os.path.isfile(item): Utilizing the
osmodule again, the
os.pathreturns a boolean after determining whether or not our item is a file.
[item for item in os.listdir('.') if os.path.isfile(item)]: Just one of the many ways to use List Comprehensions in Python, this code reads out as “For each of the items in the directory, add it to the list if it’s a file”.
- Line 4: A simple printing of the filename to the console
##Calculating the hash There are many different types of hashes to chose from, all with varying speeds and levels of security. In a production-level FIM, you’ll want to take things like calculation speed and collisions into account, but for the purposes of this post, we’ll use MD5.
- Line 1: Here, we add the
hashlibmodule so we can access the hashing functions we’ll need later.
- Line 4: Creates a new instance of
- Line 6: Because we may be monitoring files larger than our available memory, we need to break them into chunks to keep the system from halting. The
iter()function allows us to repeatedly perform a task until certain criteria is met. In this case, we are using a
lambdato read out 2048 bytes at a time, and will stop once the file reaches its end, returning
''. (Note: The reason behind 2048 is that MD5 uses a block size of 128. By using a multiple of that, we can not only read the file faster, but help to calculate the hash faster as well.)
- Line 7: Using the byte chunks gathered from the previous line, we use the
update()function to push the new chunk into the hash object.
- Line 8: Generates the MD5 hash in hexadecimal format.
##Storing Hashes So now that you’ve got your files hashed, it’s time to put them some place where you can access them later.
- Line 3: Start by declaring a new variable,
files, as an empty dictionary.
- Line 10: Python loves to make things easy. This line is actually doing two things depending on whether or not the file has already been seen. If
fileis not currently a key in
files, it is added with its value set to
filedoes exist in the keys, its value is updated to the new hash.
##Send a useful alert Here’s where you get to be creative! When it comes to alerting, you have a number of options to choose from. Customize the format, come up with a creative message, write to the console, send an email or text message, the possibilities are endless!
- Line 1: We need to import the
timemodule in order to access some date/time information.
- Line 10: To simplify this line, I’ll break it down into pieces
time.strftime("%Y-%m-%d %H:%M:%S"): Using the
strftime()function, we can pass it a string format for it to output. If you want to make your own format, or just learn more about
strftime(), check this out
'%s\t%s has been changed!'%(string, string): This is just one of Python’s many ways to format a string. By replacing
%swith string variables (
%dfor numbers), you can create strings cleanly (‘no need’+’ for ‘+’this’).
##Detecting the change Because we trust the baseline hashes and only want to be alerted when they change, we need to add some sort of check to prevent our alert from always going off.
- Line 10: If
fileexists in the keys of
files(preventing alerts on the first run), and
md5is not the same as
files[file]’s value (the file has been changed), the alert will be triggered.
####Output: After this step, you shouldn’t see anything! But that will change shortly…
##Continuously Monitor So far, you’ve scanned your directory, picked out the files, collected their hashes, and added alerts. For this final step, we’ll throw it all in a loop to keep the code running and start the monitoring.
- Line 4: Creates a never-ending loop, rechecking our files with each iteration.
- Line 14: After each iteration, we want to make sure to pause our monitoring. Without this pause, we would occasionally run into permission issues and crash the script.
##Conclusion Congratulations! You’ve built your very own File Integrity Monitor. Even though it’s very basic, all the core fundamentals are there for you to build off of. If you are interested in learning more, check back for future posts on building a more advanced FIM (along with other security related goodies).