Practice
A new set of Android 10 images and files have been contributed to the collection by Joshua Hickman.
Practice
Android 7 is the contents of a LG GSM H790 Nexus 5X phone. It is distributed as a compressed tar file called android_7.tar.gz that contains:
Practice
Android 8 is a LG GSM H790 Nexus 5X cell phone. It is distributed as a compressed tar file.
android_8.tar.gz contains:
0 Android 8/
1203 Android 8/LG GSM_H790 Nexus 5X.ufd
31268536320 Android 8/blk0_mmcblk0.bin
4194304 Android 8/blk32_mmcblk0rpmb.bin
4168 Android 8/procdata.zip
457094 Android 8/Android8-ImageCreationDocumentation.pdf
17069 Android 8/Android 8 Hashes.pdf
1789792 Android 8/Takeout Data/Twitter-TDfir.zip
36296652 Android 8/Takeout Data/GoogleTakeout-thisisdfir(at)gmaildotcom.zip
6813 Android 8/Takeout Data/WhatsApp-thisisdfir-accountdata.zip
4320066 Android 8/Takeout Data/facebook-100030861781799.zip
353055 Android 8/Takeout Data/Instagram-thisisdfir.zip
272449 Android 8/Takeout Data/Snapchat-thisisdfir.zip
You can download Android 8 from: http://downloads.digitalcorpora.org/corpora/mobile/android_8.tar.gz
Practice
Android 9 is a dump of a Google Pixel 3 phone running the Android 9 operationg system.
Practice
Practice
You will find network packet dumps in the following data sets:
- M57-Patents Scenario has packet dumps from each day of operations.
- Nitroba University Harassment Scenario consists solely of packets captured at a dorm room.
- Over 2000 packet dumps from DEFCON 20 Capture The Flag
- A 5GB TCP connection, just the thing for testing TCP/IP reassembly tools
If you are looking for network packets, you may find these pages on other websites useful as well:
- The MIT Lincoln Laboratory’s DARPA Intrusion Detection Evaluation data set.
2010-nps-emails is a test disk that can be used for testing programs that find email addresses or perform string search.
The disk image consists of 30 different email addresses, each one stored in a different document with a different coding scheme.
Below are a list of the email addresses and their codings:
email address Application (Encoding) plain_text@textedit.com Apple TextEdit (UTF-8) plain_text_pdf@textedit.com Apple TextEdit print-to-PDF (/FlateDecode) rtf_text@textedit.com Apple TextEdit (RTF) rtf_text_pdf@textedit.com Apple TextEdit print-to-PDF (/FlateDecode) plain_utf16@textedit.com Apple TextEdit (UTF-16) plain_utf16_pdf@textedit.com Apple TextEdit print-to-PDF (/FlateDecode) pages@iwork09.com Apple Pages '09 pages_comment@iwork09.com Apple Pages (comment) '09 keynote@iwork09.com Apple Keynote '09 keynote_comment@iwork09.com Apple Keynote '09 (comment) numbers@iwork09.com Apple Numbers '09 numbers_comment@iwork09.com Apple Numbers '09 (comment) user_doc@microsoftword.com Microsoft Word 2008 (Mac) (.doc file) user_doc_pdf@microsoftword.com Microsoft Word 2008 (Mac) print-to-PDF user_docx@microsoftword.com user_docx_pdf@microsoftword.com Microsoft Word 2008 (Mac) print-to-PDF (.docx file) xls_cell@microsoft_excel.com xls_comment@microsoft_excel.com Microsoft Word 2008 (Mac) xlsx_cell@microsoft_excel.com Microsoft Word 2008 (Mac) xlsx_cell_comment@microsoft_excel.com Microsoft Word 2008 (Mac) (Comment) doc_within_doc@document.com Microsoft Word 2007 (OLE .doc file within .doc) docx_within_docx@document.com Microsoft Word 2007 (OLE .doc file within .doc) ppt_within_doc@document.com Microsoft PowerPoint and Word 2007 (OLE .ppt file within .doc) pptx_within_docx@document.com Microsoft PowerPoint and Word 2007 (OLE .pptx file within .docx) xls_within_doc@document.com Microsoft Excel and Word 2007 (OLE .xls file within .doc) xlsx_within_docx@document.com Microsoft Excel and Word 2007 (OLE .xlsx file within .docx) email_in_zip@zipfile1.com text file within ZIP email_in_zip_zip@zipfile2.com ZIP'ed text file, ZIP'ed email_in_gzip@gzipfile.com text file within GZIP email_in_gzip_gzip@gzipfile.com GZIP'ed text file, GZIP'ed
The image can be downloaded from http://downloads.digitalcorpora.org/corpora/drives/nps-2010-emails/
The submission contains four raw (dd) image files of the USB flash disk «Transcend JF V10 / 1GB, D33193», two packet capture (pcap) files and four log files. The disk is non-partitioned and contains no file systems; it contains many non-deterministic sectors (each sector contains 512 bytes).
Namely, each sector that doesn’t belong to a written block of flash memory cells contains non-deterministic data (instead of null bytes, as many forensic examiners tend to expect). The disk does function properly though. Several tests show that writing to a sector turns its contents to deterministic state (i.e. you will read exactly what you wrote).
Days were spent to understand why there are non-deterministic blocks of data. The study showed that each non-deterministic sector represents the contents of the SCSI READ(10) command related to reading that sector. In other words, when the disk receives SCSI READ command that covers non-written sectors it simply sends the contents of the command back to the host, and these contents appear as sector data to an operating system.
In the experiment two raw images of the USB flash disk were acquired on a Linux host using dc3dd (these image files together with corresponding dc3dd log files can be found in «linux-dc3dd/»), and two other raw images were acquired on a Windows 7 host using FTK Imager (image files and log files are located in «windows7-ftkimager/»); all images have different hash values. Windows host was also running capture software to intercept all USB commands and replies, this data was written to pcap files named «usb-1» and «usb-2» (for the first and the second acquisition accordingly). There were no writes to the disk during or between acquisitions. The disk was disconnected between acquisitions on a Windows host: this was done to assign a new tag to the command blocks of all SCSI READ(10) commands going to the disk (unlike Linux, Windows uses the same tag in the command block of all SCSI READ(10) commands, the tag seems to be generated randomly when a disk is connected via USB; Linux, conversely, assigns new tag to every command block of SCSI READ(10) command); otherwise, two images would have the same hash value on a Windows host (results of hashing the disk twice without reconnecting it are shown on the screenshot located at «windows7-ftkimager/ftk-imager-screenshot.png»).
Let’s look at the sector #100005 in four images acquired (dd options: skip=100004 count=1).
«linux-dc3dd/flash-firstrun.dd» has the following data:
00000000 55 53 42 43 94 06 00 00 00 80 00 00 80 00 0a 28 |USBC...........(|
00000010 00 00 01 86 80 00 00 40 00 00 00 00 00 00 00 60 |.......@.......`|
00000020 00 60 ff ff ff ff ff ff ff ff ff ff ff ff ff ff |.`..............|
00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200
«linux-dc3dd/flash-secondrun.dd» has the following data:
00000000 55 53 42 43 00 7f 00 00 00 80 00 00 80 00 0a 28 |USBC...........(|
00000010 00 00 01 86 80 00 00 40 00 00 00 00 00 00 00 44 |.......@.......D|
00000020 00 44 ff ff ff ff ff ff ff ff ff ff ff ff ff ff |.D..............|
00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200
«windows7-ftkimager/flash-firstrun.001» has the following data:
00000000 55 53 42 43 d8 b1 6b 91 00 40 00 00 80 00 0a 28 |USBC..k..@.....(|
00000010 00 00 01 86 a0 00 00 20 00 00 00 00 00 00 00 5a |....... .......Z|
00000020 00 5a ff ff ff ff ff ff ff ff ff ff ff ff ff ff |.Z..............|
00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200
«windows7-ftkimager/flash-secondrun.001» has the following data:
00000000 55 53 42 43 20 6a 7f 83 00 40 00 00 80 00 0a 28 |USBC j...@.....(|
00000010 00 00 01 86 a0 00 00 20 00 00 00 00 00 00 00 79 |....... .......y|
00000020 00 79 ff ff ff ff ff ff ff ff ff ff ff ff ff ff |.y..............|
00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200
As you can see, sectors are slightly different. Now let's
dissect the data in the first hexadecimal dump (using various colors to
highlight the bytes):
00000000 55 53 42 43 94 06 00 00 00 80 00 00 80 00 0a 28 |USBC...........(|
00000010 00 00 01 86 80 00 00 40 00 00 00 00 00 00 00 60 |.......@.......`|
00000020 00 60 ff ff ff ff ff ff ff ff ff ff ff ff ff ff |.`..............|
00000030 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00000200
Letters «USBC» point us to the USB command block, which starts with four-byte signature «USBC». The structure of the USB command block is presented below (taken from Linux kernel source code):
struct bulk_cb_wrap {
__le32 Signature; /* contains 'USBC' */
__u32 Tag; /* unique per command id */
__le32 DataTransferLength; /* size of data */
__u8 Flags; /* direction in bit 0 */
__u8 Lun; /* LUN normally 0 */
__u8 Length; /* of of the CDB */
__u8 CDB[16]; /* max command */
};
The CDB field contains an actual command transmitted. The first byte of the CDB field is 0x28, which refers us to the SCSI READ(10) command, which operational code is 0x28 (see Table 85 in the «SCSI Commands Reference Manual» by Seagate: www.seagate.com/staticfiles/support/disc/manuals/scsi/100293068a.pdf). SCSI READ(10) command is exactly ten bytes in length (not counting previous USB command block header).
struct read_10 {
__u8 opCode; /* operational code (28h) */
__u8 Flags; /* various flags */
__u32 LBA; /* logical block address (MSB first) */
__u8 Group; /* group number */
__u16 TransferLength; /* transfer length (MSB first) */
__u8 Control; /* control byte */
};
The contents of SCSI READ(10) command are: logical block address is 18680 in hexadecimal, or 99968 in decimal; transfer length is 40 logical blocks in hexadecimal, or 64 in decimal. Note that we were analyzing sector #100005, which is between 99968 and 100032 (99968+64). Now let’s check what data is present in the sectors #99967 till #100032 (one-liner for bash: «for i in `seq 99967 100032`; do echo -n “$i: “; dd if=flash-firstrun.dd skip=$i count=1 2> /dev/null | md5sum; done»): sectors #99968 till #100031 have the same data as sector #99968; sector #99967 differs from them, as well as sector #100032. The conclusion is that sectors #99968 till #100031 have non-deterministic data, which represents the contents of the SCSI READ(10) command used to read that sector range.
A program was written to study all non-deterministic sectors the same way as described above, and the results are the same — every non-deterministic sector contains a corresponding SCSI READ(10) command.
The data can be downloaded from:
http://downloads.digitalcorpora.org/corpora/drives/nps-2014-usb-non-deterministic/
Related links
1. https://www.forensicfocus.com/articles/flash-drives-and-acquisition/ (Flash drives and acquisition by Dominik Weber)
2. https://www.forensicfocus.com/forums/general/fat32-strangeness/ (FAT32 strangeness by «Fab4»)
Practice
Packet dumps on this server:
- Nitroba University Harassment Scenario
- Daily packet dumps part of the M57 Patents Scenario
- DEFCON 19 (2011) and 20 (2012) CTF packet dumps
Packet dumps from other organizations:
- DARPA 1998 Intrusion Detection Evaluation Data Set [homepage]
- DARPA 1999 Intrusion Detection Evaluation Data Set [homepage]
- DARPA 2000 Intrusion Detection Scenario Specific Data Sets [homepage]
As one of the most widely spread database systems in the world, SQLite is used on an immense number of computer systems. A vast number of software programs like browsers or smartphone apps are using SQLite3 databases to store application data. In many cases, such data is of high value during a forensic investigation. Therefore, various tools have been developed that claim to support rigorous forensic analysis of SQLite database files. Such claims are not supported by appropriate evidence, as standardized collections of databases were long missing. These can be leveraged by the forensic community for purposes like testing, validating, comparing and improving such tools.
We present a standardized corpus of SQLite files that can be used to evaluate and benchmark analysis methods and tools. The corpus contains databases which use special features of the SQLite file format or contain potential pitfalls to detect errors in (not only forensic) software. The corpus has also been enhanced by anti-forensic aspects within specifically crafted databases. These do not necessarily conform to the SQLite file format specification and can thus be used to additionally challenge the extraction and recovery routines of (forensic) tools. We call the database collection “SQLite Forensic Corpus” and donate it into the public domain.
A link to the corpus can be found here
The Articles linked to the corpus:
Sebastian Nemetz, Sven Schmitt and Felix Freiling, “A standardized
corpus for SQLite database forensics”, Digital Forensics Research
Workshop Europe, 2018
https://www.sciencedirect.com/science/article/pii/S1742287618300471
Sven Schmitt, “Introducing Anti-Forensics to SQLite Corpora and Tool
Testing”, IMF 2018, 11th International Conference on IT Security
Incident Management & IT Forensics
https://www.computer.org/csdl/proceedings-article/imf/2018/663200a089/17D45WnnFUl
Practice
The Real Data Corpus (RDC) is a collection of raw data extracted from data-carrying devices that were purchased on the secondary market around the world. Many studies have shown that hard drives, cell phones, USB memory sticks, and other data-carrying devices are frequently discarded by their original users without the data first being cleared or purged. By purchasing these devices and extracting their data, we have created a data set that closely mimics data as it is found in the real world.
Potential Uses
The Real Data Corpus is a one-of-a-kind scientific resource for:
- Developing and validating forensic and data recovery algorithms and tools.
- Developing and validating document translation software.
- Exploring and characterizing real-world computing practices, configuration choices, and option settings.
- Studying the storage allocation strategies of file systems under real-world conditions
Current Contents
- A total of 156 hard drive images ranging in size from 500MB to 80GB.
- Approximately 600 flash memory images (USB, Sony Memory Stick, SD and other), ranging from 128MB to 4GB.
- Approximately 100 CDs, all purchased outside the US.
- Approximately 10 digital camera memory images.
- Approximately 40 GSM SIM chip memory images.
More details of the corpus content can be found in Garfinkel, Farrell, Roussev and Dinolt, Bringing Science to Digital Forensics with Standardized Forensic Corpora, DFRWS 2009, Montreal, Canada.[1]