Following up on a forensic artifact project database idea , the end result is that the idea is dead before it started.
The twitter poll (one of the most unscientific, but easiest polls to do) didn’t show a lot of promise. Also, there were a LOT of DMs and email discussions. Thanks to everyone giving me their thoughts.
Here are the main points that I received, summarized in three statements:
-Publishing research must be in academia (journals)
-Publishing research must be in books (publishers)
-We don’t need project management in research
On top of these points, the fear of lack of contributors holds me back. According to the Twitter poll, less than half (of only 88 who voted), would contribute. That is not quite a big enough number in percentage or actual number given how many people actually do research and compared to the discussions I had with some very passionate folks.
Some suggestions given were;
-Those who have published and those who have done research should connect with each other to publish the research.
-Those who do research should go through the academia route to publish in journals.
I don’t see this happening to any great degree, other than a perhaps a handful of instances.
We are left to relying on DFIR/Infosec bloggers for the most current research, which will have to make due for citable sources that do not exist in publications. Ironically, this was the original tweet concept that started the conversation to begin with.
As to immutable citation sources, we still have books and journals, and everything else will be dynamically changing and evolving, which is a double-edged sword. Good in the sense that we have nearly instant access to the newest developments via blogging. Bad in that blogs are not peer-reviewed, nor immutable. Blog content changes, which can make for a confusing citation. Blogs also disappear without notice, which again, affects citations.
I do foresee a time where a practitioner will be able to more quickly publish a peer-reviewed and community accepted work outside of pure academia, but unfortunately, it is not today. The peer-review process, as it stands in academia, is a long process and requires probably more time to finally be published than it did for the actual research. This should be opposite, but it is what it is. Most importantly, however, is that the DFIR/Infosec blogs are awesome for the most up-to-date, practical, and useful research that exists on the planet. Do not discount any research that was personally conducted by a practitioner. It may be right. It may be wrong. Regardless, each is a nugget of gold to expand upon and personally validate in your own research. For that, if you are a DFIR/Infosec blogger, you have my respect.
Thanks to all who contributed their opinions!
A weekend Twitter thread about having a lack of citable, peer-reviewed, DFIR research prompted me to volunteer to host a project management website (a sub-domain of dfir.training). I think the need is real for reasons mentioned on the Twitter thread, but whether or not it can work is all together a different matter.
From what I have seen, peer reviewed DFIR research generally lives within journals and books , or within the walls of academia . Either the research is not freely available and/or not easily found within the walls and halls of educational institutions. Research is blogged about, presented at conferences, and uploaded to the Internet via any number of websites, with much of this work not being peer reviewed . There is too much great effort that is never formally published in which the researcher deserves to (1) receive formal recognition and (2) be formally peer-reviewed by the community without having to be published in a journal or book.
I absolutely give credit to the bloggers sharing their research online, more than you can imagine and for many reasons. However, referencing a blog in a case report, affidavit, or in a research paper does not quite reach the level of peer reviewed research as a source of information. The life of a blog is also indefinite, dynamic in nature, and many times never found by those who need the information.
Given enough interest, I will gladly maintain the website, and ensure that the research will always be freely available. I will manage the users and research groups to reduce the risk of trouble-makers (such as bots and spammers) from having access to the editable parts of the research projects.
Playing devil’s advocate, here is what I see:
-Lack of willing contributors
-Lack of willing peer-reviewers
-Continued reliance on wiki’s and blogs as a source of non-peer reviewed research
-Continued non-sharing of personal research
Countering this, I see:
-Contributors being those who have already completed personal research, who can now have their work peer-reviewed
-Peer reviewers growing professionally by helping and mentoring researchers
-Reliance on credible, peer-reviewed, professionally monitored research for citable resources
- You personally being credited and formally validated by the community for your work
-Research that is developed, peer-reviewed, and published months faster than in a journal or book
-Research that meets Daubert-Frye standards (community accepted methods)
- You get another source of community validated research without having to pay for it
I have a Twitter poll that will expire in a few days. If not enough interest, perhaps the timing is not right. Personally, I think it is past time for an additional means to create peer reviewed research for those who would rather see their efforts received by the community-at-large, rather than kept behind paywalls or within privately accessible collections of research.
As to the mechanics of how this can work
-Project manager (researcher) initiates a research topic
* Project created
* Tasks created
*Contributors join in the project
*Project manager keeps the research going through final draft
*Public does not have access to the active research (unless they want to contribute)
-Final drafts reviewed by peer review process (any number of types, such as a blind review, double-blind, etc…)
*Corrections, suggestions, recommendations made
*Sent back to project team
-Project team makes corrections as necessary (or defends any claims against research)
-Final peer review and publish (accessible to the public)
All of the above is through a project management platform. The final peer reviewed and approved project would be in a standard format. This concept is to provide an additional means of peer review for that research which currently sits on websites without earning documented community credibility.
Imagine taking research you have done in the past, perhaps it's on your blog, and you get it peer-reviewed, meets Daubert-Frye standards, and becomes available to everyone as a credible source of DFIR information. That's the point to all of this.
Or, we can stick to citing blogs and wikipedias…
I was speaking to someone at Infosec Europe last week about ‘getting into this field of infosec’. I kept answering all questions with the same answer of telling the guy to get started and do something. But the future DFIR’r kept telling me about all the training and schooling that he had completed, the training and schooling that he is planning to do, and what to do next. I was quite impressed with how much training already done, including earning a degree and having taken a dozen vendor courses. I was disappointed in how much more he was planning to do before ever starting work in the field that he has spent years in learning, but not doing.
In short, I told him to stop his training and education right now and make this conference his last until he puts to work what he has learned so far.
He was stuck in learning mode, repeating courses and conferences over and over again with the expectation that competence will come to him. This is a terrible mistake. There is a point where you have enough training and education to start, where you can do the most basic of job tasks, and where you can apply what you already know to learn more through experience. To be stuck in learning mode is to never know what you can through practical applications. Testing, theories, and essays are only part of the equation in becoming competent.
Don’t get stuck in learning mode. This applies even if you are competent in your field with experience. At some point, which is different for everyone, spending more time in education isn’t going to propel you farther than if you are actually doing the work, practicing that what you know, and discovering what is not being taught in class. Avoid the point of diminishing returns on “learning” when you have more than enough to be “applying” what you have already learned.
I’m not advocating to never attend a conference or training course, or not to get an advanced degree. And I am not saying that experience is better than education. I am saying that there is a balance needed between education and experience. Having a balanced portfolio of experience, education, and continuing learning builds your competence base much better than focusing solely on the academic or solely on the experience aspect of DFIR or any field for that matter.
When I say, “Get out of the learning mode!” , I am saying that in the manner of take what you have learned formally and put it to use physically. You will still be learning, but you will be learning differently, and learning things you won’t learn in a formal training atmosphere. You will learn by doing, which will make your future formal learning that much more effective because you have had your hands on the things being talked about in a classroom.
I used to sarcastically joke in my police days that some cops seem to be in training every other week and had taken so many training courses that the only training left was underwater handcuffing classes. Training and formal education, much like training wheels on a bicycle, is to get you up to speed to start working on your own where you can excel well past what any training wheels can do for you.
Break the Groundhog Day cycle. Put your knowledge to work and complete the path to competence by learning AND doing.
Like many others working in DFIR, I occasionally get asked questions on how to get a job in DFIR. By DFIR, I mean the overall field of digital forensics/incident response/electronic discovery. Sometimes, the questions are loosely asked as if it is easy to get in by someone who thinks they are 'good with computers'. Other times, I am asked by those with computer science graduate degrees and tons of computer experience. The range is quite wide. I am certain that anyone and everyone looking to make a break into DFIR has already Google'd it, found a lot of blog posts, and still are having a difficult time getting in the door. That is just the way it is. Employers feel like they can't find anyone and everyone feels like they can't find an employer to hire them.
I have blogged about this before, and I'm writing again because this is not only a common topic, but it is really important if you are trying to get a job in DFIR.
I wrote a short paper on my thoughts, ideas, and opinions that you can download by subscribing to the DFIR Training newsletter . Yes, I am sneaking in the newsletter to get the download, but you can always unsubscribe if you don't like what I do with the monthly newsletters. I am betting that you will get a benefit of the newsletter.
As far as to giving another opinion on how to get into DFIR, I believe that the more people that give their opinion and experiences on how they got in, the more likely that someone will be able to use that guidance. We each have had different paths to "make it". Some of us had no help, others had a little help, and a few fortunate few had a lot of help. But everyone who is here can easily lend a hand to the new folks coming up by giving a few words of positive advice.
Oh yeah, I created a "Get Hired for DFIR" checklist on the download for those who like checklist. I am one of those who use checklists, which is why I made the checklist :)
For those working to get into DFIR, you can do it and we are waiting for you to check the boxes and get hired!
All right. You got me. I am not going to force you to subscribe to get a download. So you can download the file here: Unlocking the DFIR Door . But I think you should subscribe anyway. The newsletter will be awesome!
I am starting a monthly newsletter to supplement the existing newsletters that many other DFIR contributors are creating. I intend to make the newsletter different enough to justify having one email a month in your inbox; actually, I intend to create an awesome monthly newsletter.
I initially expected to give this a shot for a month or so and then see if the signups would justify the effort to create a newsletter. MailChimp has a ‘forever free’ plan that looks to fit what I’m looking for so for testing the water for interest, MailChimp it is then. However, after just a week, the signups are reaching the limit of the free plan. No biggie as this is a pleasant surprise to see the interest. The first newsletter won’t come out until June, but you can sign up now so that you don’t miss it.
A DFIR Training Helper!
One of the things happening at DFIR Training is me getting a helper! This should make updates to the website go a little faster (faster additions of artifacts, tools, etc…). Plus, I will have time to catch up on the projects that keep getting pushed farther and farther behind.
Upcoming site features
There will be an opportunity for guest blogging at DFIR Training. The purpose will be to give traction to anyone wanting their research to be seen by more people. My suggestion is that if you have a blog that you want help marketing, either write a unique post or cut-n-paste something from your blog for a guest post (and link DFIR Training to your blog). The goal is more exposure for research and more credit for the work you do.
Other website additions are coming as well, which I’ll keep under wraps until I actually start them once my helper gets more fully on board. Expect to see uniquely created content soon enough.
DFIR Training is popular because…
….it has what you want on it. Visitor stats are high and the site ranking has trended upward fairly well. Page views and data transfer is astronomical ( 90GB of bandwidth and 4.5 million hits in 2017). I suspect this to go higher as soon as I start adding the new stuff over the summer.
I put up more stats here if you are interested: About DFIR Training
What else do you want to see?
If there is something you want on DFIR Training, send me a note and let me know. Complaints, compliments, and suggestions are welcome if the intention is to make the site better for the community. In other words, let’s be nice ?
It has a lot of DFIR stuff
It really has a lot of DFIR stuff
Even more DFIR stuff is coming!
I checked the stats for dfir.training for the months of April 2016, April 2017, and April 2018 to get a gauge on what pages are most popular, where most visitors are coming from, and areas to focus on content (based on the pages and page behavior flow). What I found is that the stats have dramatically increased since April 2016, particularly toward the end of 2017 (The stats do not include bot traffic, which would unnaturally inflate the hits). My conclusion is that so far, everything seems to be in order since the number of visits and hits are higher than ever before.
Even tho the visit rate is high, I am still planning to add more content (new stuff!) that I have been thinking about but haven't had the time to do yet. The new stuff is in addition to the regular updates and content that needs to be added incrementally, like the forensic artifacts and tools. I will have a website 'helper' later this year so that the amount of updates and data entry will be twice as fast as it is now.
With that, and considering the number of visits.......if you are one of the dfir.training visitors and you find the site useful, your vote for DFIR Training as Digital Forensic Resource of the Year would be appreciated.
Hurry as the last day to vote is coming up quick (May 25).
As to the stats, here are a few stats from April 2016, 2017, and 2018.
I visited a DFIR shop and as I was leaving, I asked one of the most experienced examiners in the shop, “ Hey, how do you like *x* tool new functions ?” and the answer was “ Never heard of *x* tool .” For me, I use *x* tool often and assumed that everyone else does, or at least knows about it. I was wrong. (I am not naming the *x*, but it can apply to any tool you use). If you don’t know that a tool exists, you are not going to use it.
The DFIR.training tool database
My opinion is that looking for a DFIR tool that does a specific thing that you need for a specific analysis is either easy or impossible, depending on how you look at it. If you only use the major name brands, you have an easy choice (because you only want to use a major name brand for everything). If you are looking for a very specific tool to do a very specific thing, you may have no choice because of being unable to find it. So, the DFIR.training tool list contains a lot, because maybe that one little tool you need might be there, sitting under the category you are looking.
Top 10 Lists
Often times, I find online lists of forensic software that are ‘best’ for your lab. You don’t have to look far to find lists that are something to the effect of the top ten free tools or the ten tools that you must have in your forensic toolbox. You can find suggestions, opinions, and even detailed scientific methods on how to choose a forensic tool.
Some warnings on pre-defined tool lists.
-They are not personalized to you. Others created the lists, based on their opinion and needs, or what they think you need.
-Lists are limited. A list with “The Top 10..” may not fit your needs, but maybe #11 would, if it were on the list. Maybe #33 would be #1 for your needs, but #33 won’t be on someone else’s Top 10 list.
-Lists can be irrelevant. How can you compare RegRipper with EnCase on the same list? Dissimilar tools compared with each other makes for an irrelevant list, but it is common to see.
When I see a list, I only look to see if there is a tool that I never heard of before, not that I accept it to be put on my personal Top 10 list because someone else says it should be.
If you don’t know that a tool exists, you won’t ever use it, even if it would be perfect for your needs. I have found some gold in Github on more than one occasion, simply by searching Github in hopes of finding something that I need but can’t find elsewhere. This takes a lot of time, but you can’t expect that a tool on Github has a marketing budget to get the word out. You have to search for it..if you have time.
Commercial companies have marketing budgets and marketing operations. If they did not market their products, few would ever hear about them, fewer would purchase them, and eventually that company closes down. This is a loss for everyone. Word of mouth only goes so far. Consider that I found a few tools in Github that could be huge commercial successes, but without substantial marketing, won’t make it into mainstream DFIR. Can you imagine if no commercial tool was ever marketed? Where would you be right now with your tools if there was no marketing?
The Menu Method of Finding the Best DFIR Tool
Rather than a scientific method, or picking from a pre-defined list, I look at DFIR tools like I look at a menu. If I am looking for breakfast, I look first at the breakfast menu for something that I may want. Sometimes, I might choose from the lunch menu for breakfast because the lunch menu is what I want at that particular time. The menu is simply an offering of things from which to choose. Just as one food can be served at breakfast or dinner, one DFIR tool can be used for collection or analysis. The menu is simply a guide.
If a particular menu doesn’t have what I want, I go to another restaurant and look at a different menu. And if none of the menus have what I want, I have to learn to cook it myself. The menu options in DFIR are your categories. Some tools fit neatly in one category, others fit in several categories.
Your Top 10 DFIR Tool List
Looking for DFIR tools work the same way as deciding what to eat for lunch. You consider your wants and needs to make a decision. I am sure that you don’t decide what you want for lunch for the rest of the year. You probably make a different decision, or at least go through the same decision-making process every day, because every day is different.
DFIR tool selection works the same. Every case is like deciding what to eat for your next meal. What kind of case is it? What do I need to do with the case? What tools will best do that job in the case? Which tools do I prefer and do these tools match with the best tools for the job? Unless you are tied to a specific tool for some reason, the choice is yours to make and not someone else’s.
Last week, while tech editing/reviewing a chapter in a book that I believe is destined to be one of the most widely used books in digital forensics, I read a short but important point: ‘know what you want to do before you start’ (paraphrased), along with an example of making this point. Perhaps this simple suggestion in forensic work is way understated.
Over the past years when I started getting into ‘computer’ forensics at the time when the resources of information were slim, training practically non-existent, and the tools far and few between for much of a choice to use, looking for evidence was pretty much going on fishing trips in data. For the training courses I did attend in the beginning, the most common approach taught was to;
-Take a full image of everything
-Ingest the images into “name-your-tool” of which you had only a few choices
-Take all the data and process it, index it, sort it, extract it, view it, search it, filter it
-Find the evidence from what you processed by looking at virtually everything…
This method doesn’t work today. The amount of data is too much. The common hard drive was less than 50GB way back then, but today you can rarely find a laptop with less than 500GB. Today’s tools are certainly capable of processing this data way more efficiently than the tools (or the versions) of yesterday. But even being able to process data faster only means you have more data to fish through in attempts to find evidence. I don’t remember the last case where I had less than 1TB of data that potentially held evidence.
We have come a long way in training today including improved processes. When given terabytes of data and asked to find the evidence, no longer do we expect that the terabytes of data to examine will turn into four or ten times the size after we ‘process’ it, because we do it better now. We are smarter than before. We ask better questions. We know more about where the evidence lives within the data. We have demanded tool-makers develop tools that pinpoint exactly what we are looking for in a quick and efficient manner. We now;
-Ask “Specifically, what is the problem?”
-Target the places we know the evidence to that problem lives
-Use tools that are narrowly specific to what we want to do
-Follow the evidence we find (one thing points to another, etc…)
-And solve the problem (find the evidence, validate it)
I write this because I still hear requests to ‘ find something on that computer to make this case ’, and each time I kindly remind that fishing for evidence in a hard drive is not only expensive in both time and money, but unproductive without targeting the problem * .
*A problem could be finding a specific user created document, a downloaded image, or an unauthorized access to a computer system.
The point to all of this being, before you push that button, or hit enter on a command line, or even connect a write-blocker to a hard drive, first ask yourself, “what is it that I want to accomplish”. The next thing you do may either give you weeks of work in vain or solve the problem before dinner time.
We have decision-making in every aspect and at every step of a forensic analysis. When we find something important, such as a user created file, we have decisions to make as to what to do next. We follow the clues in order to determine what happened on the machine.
“What ‘bad’ things happened?”
“When did the bad things happen?”
“How did the bad things happen?”
“Who did the bad things?”
For the most part, we had it easier back in the early days of forensics. If evidence was found on the hard drive, then it was the person who possessed the computer that was the suspect. At a certain point in time, physical control of computers did not necessarily mean that the possessor of the computer was the suspect. Remote access via any number of methods (pick your method of installing malware) meant that not only do you know how to find the evidence, but you also have to find a suspect that potentially may not be the owner of the computer. At the very least, you need to ensure that the owner of the computer is not the suspect if that is the case.
As simplistic as this sounds, it is not.
Without getting into the pitfalls of investigative work (including forensic analysis), one of the best methods to avoid falling into a rabbit hole is to keep asking yourself questions as you find evidence in data. Finding a LNK file of importance may be critical in a case, but for all the wrong reasons. Perhaps the LNK file shows that a user physically clicked to view a ‘bad’ website, which might imply intention and knowledge. Or perhaps the LNK was an attack vector , which caused the computer to be compromised and controlled by someone else. In one scenario, the computer owner is the suspect , but in the other, the computer owner is the victim . The consequences for getting this wrong will be devastating for you and the victim.
This is the crux of any investigation. Catch the right person. Don’t pursue the innocent person.
Identifying the crime is step 1. Finding evidence is step 2. Finding the right person is step 3. There is a huge and varied range between #2 and #3. Sometimes it is easy and sometimes it is not. When it is really difficult, the worst result is actually not not finding the bad guy but believing the victim to be the bad guy. As an example, finding evidence on a computer that was committed by a remote malicious actor, but blamed on the computer owner, is worse than not being able to identify the suspect at all. This can happen in any type of case, but the risk in computer crime can be so much easier to do.
As you dig your way through data and find tidbits of clues and evidence, you have to figure it out.
Don’t assume. If you plant a seed in your head at any point in an analysis, it will grow. Pigeon-holing your theory during an analysis will result in your assumptions being your end result regardless of what may have really happened on the computer. You will see facts fit preconceived beliefs more than letting the facts show you the truth. You have decisions to make when following the evidence, so base the decisions on you what actually see as evidence. In order to make good decisions, you have to do a thorough job, and keep questioning the evidence you see and the manner in how you see it. You want to get cases right. You have to get cases right.
When you work with a lot of forensic tools, there is never a single “that time of year” to renew your annual maintenance fees as it feels like “that time of year” is every month. Mind you, I’m not complaining one bit, but I did have a conversation today over coffee about the cost of forensic software and listened to a lot of complaints about the cost of the DFIR business with my fellow DFIR buds.
Most of the complaints stemmed from the high initial cost of software, with hardware complaints taking a close second place, and the maintenance fees a distant third place. Interesting enough, we each knew the current going rates for just about everything because when you have to write big checks*…you tend to remember how much the checks were written for…
Some of the complaining I heard was summed up with:
I agree that the tools are expensive if you compare them to other things you buy during your lifetime. Some tools, like the monster forensic workstations, cost as much as a compact car! But in the grand scheme of things, I think the prices are reasonable for a few reasons. For one reason, you can’t do the job without a set of tools. FOSS tools only do so much and commercial tools only do so much, so you need a toolbox with both FOSS and commercial tools to get the job done. Yes, you can do theoretically do everything with open source tools and not spend money on software, but realistically, that is not always possible all of the time for all of the folks working in DFIR.
For the business proprietors who directly write the checks, any expense is too much because it takes away from business revenue. When the expenses are in the tens of thousands, the checks are quite heavy; however, I tend to counter complaints of cost with a question of “How many cases do you have to work in order to pay for one license?” The answer varies, but sometimes one decent case can pay for a year’s worth of software, hardware, and training. When you look at the revenue earned with a software license and computer, which the revenue is replicated again and again over the lifetime of the license, the costs are not that bad. Depending upon how many cases you do and how much you or your company charges, the ROI on the tools are actually really good.
Early in my private forensic work, I quickly learned to add up the expenses for tools and compare it to revenue. No algebra needed. No complex effort either. I simply subtracted the annual cost of the tools from the annual billable hours where I used each tool. Easy enough to get a high level view of tool ROI.
Now, I’ve only talked about the private DFIR folks, but I also personally know how it works in government. The biggest differences are ‘who is writing the check’ and that there is no revenue with government forensic work to easily justify the purchase and renewal of forensic tools. But you can use ROI in the manner of work hours, effectiveness, and efficiency. With the proper tools, government examiners can more quickly get more cases done as well as being able to handle more types of casework. I know of agencies that don’t certain types of cyber crime cases simply because the agencies refuse to pay for the tools or training. That’s not good on any level but should be an easy sell by the examiners. It would be like a police department not being able to investigate robberies because no one has the tools or training to do it.
The same philosophy goes toward training. I absolutely agree that some training is extremely expensive, especially when you add in travel, meals, and lodging. However, the same thinking also applies to ROI. Learning a skill can be applied across one or all cases you have for the rest of your work career. So, if you learned a skill that for years you billed clients on numerous cases, the ROI was well worth it. Even with government examiners, a skill learned from a course can solve a case, make your day in court, and give victims justice . Without training or education, you won't be able to do some things because you just won't know what you don't know.
My coffee time today ended well, not because I convinced my coffee buddies that we actually have a reasonable cost for tools, but that I got to spend some time talking shop with a group of people who are all working in a really cool field. You can’t put a price tag on that.
*by writing checks, i mean entering your credit card information online....