Ontrack Data Recovery Solution’ Contents:
1. Data Loss Dangers
2. Critical Data Risks
3. Challenges and Responsibilities
4. Protection, Detection, Correction
5. Finding the Optimum Solution
6. The Data Recovery Alternative
7. When is Data Recovery Necessary?
8. Data Recovery Case Studies
9. Final Analysis
10. Additional Information
URL: Ontrack Data Recovery Solution
Microsoft® SQL server is a business enterprise software package that manages data through a client/server relational database. There are four fundamental aspects to a SQL (Structured Query Language) server.
1. Database File
2. Relational Database Concept
3. Client/Server System
4. Database Management System (DBMS)
The Database File refers to the physical file that contains the data the SQL server manages. This file is more than just a container for data. SQL files, or .MDF files, are highly organized and complex. The file is designed like this so that SQL server can handle multiple data transactions at once.
The Relational Database Concept is the method employed to organize the data; this is separate from the internal organization of the file itself. Relational Database Concept systems use mathematical set theory to organize the data in the most effective way. Data organization is critical to maintaining the speed of the database server.
The Client/Server System refers to the part of SQL server that communicates with the operating system. This part of the system needs to manage the server’s resources, connections and managing multiple databases at the same time.
The Database Management System interacts and works will all the above items. Not only does this manage the internal structure of the .MDF file, DBMS also controls the Relational Database side of things and organizes the data. The DBMS of SQL server is the heart of the system.
Data loss situations-What can happen?
Data loss situations can occur at a number of different levels. The first level of data loss starts with the storage device. This can be a single hard disk or a SAN or RAID storage array. The second level of data loss can start at a file system level. The file system is a unique method of storing and organizing system and user files. The operating system controls the file system. If the information about where data files are on the volume are damaged or lost, then adjustments or repairs need to be completed at a file system level. Once the file system is repaired then the file’s data stream can be accessed correctly. Most operating systems have a utility that will automatically fix the file system. These utilities work to make the volume accessible. However, this can permanently damage the data stream of the file. The third level of data loss can occur within the file itself. As previously mentioned, .MDF files are quite complex. The internal structure can be corrupted to the point where the DBMS will not make the database available.
The solution to data disasters
The process of a SQL recovery begins with finding the database file requested. If the database file is inaccessible due to problems with the storage device, then should work to get that data extracted. If the file is missing, deleted or truncated due to problems with the file system, then should work to adjust or repair the file system to correctly point to the data stream of the file. And finally if the file system has no pointers at all to the file, then should search the entire drive looking for SQL data.
The next stage is to work on the database file itself. This is where the SQL toolset comes. These tools analyze the complex internal structures of the .MDF file and provide reports as to what tables are recoverable. The reports also list the number of data rows that will come back.
Once the evaluation is complete and authorization is given to recover the data, the SQL toolset will copy the data into another SQL server database. After this phase is complete, we can backup or archive the recovered data.
Remote Data Recovery is the fastest solution for SQL recoveries. The Remote Data Recovery process will copy the recovered data into a SQL server that is setup on you or your client’s site. They only need SQL server running. For best results I recommend that the storage device we are working on be connected to a SQL server, because all of the data transactions will be occurring on a single machine.
Overview of RAID
The heart of the RAID storage system is controller card. This card is usually a SCSI hard disk controller card (however, IDE RAID controller cards are becoming quite common). The task of the controller card is to:
1. Manage Individual Hard Disk Drives
2. Provide a Logical Array Configuration
3. Perform Redundant or Fault Tolerant Operations
RAID History
RAID is an acronym for Redundant Array of Inexpensive Disks. The concept was conceived at the University of California, Berkeley and IBM holds the intellectual patent on RAID level 5. The University of California, Berkeley researchers, David A. Patterson, Garth Gibson, and Randy H. Katz worked to produce working prototypes of five levels of RAID storage systems. The result of this research has formed the basis of today’s complex RAID storage systems.
Individual Drives Management
The RAID controller will translate and communicate directly with the hard disk drives. Some controller cards have additional utilities to work with the disk drives specifically, such as a surface scan function and a drive format utility. In the case of SCSI based cards, these controllers will provide additional options to manage the drives.
Logical Array Configuration
The configuration of the logical array stripes the data across all of the physical drives. This provides balanced data throughput to all of the drives—instead of making one drive do all the work of reading and writing data, now all of them are working together and the data is streaming across all of the physical drives.
The Operations to Redundant or Fault Tolerant
The redundancy in a common RAID 5 configuration is the result of using a Boolean mathematical function called Exclusive OR (XOR). This is commonly referred to as Parity. The XOR function is a logical binary process—its best to think of Parity as combination of the other drive’s data blocks. Every byte that gets written to one data block is calculated against the other data blocks and resultant Parity is written to the Parity block for that particular stripe. What makes this function so unique is that the math will always work regardless of what data block is missing. However, the limitation to RAID 5 is that only one data block can be missing—the math will not work if there are two blocks missing. In the working environment this means that only one drive can fail. The RAID 5 configuration will not provide proper redundancy if two or more drives fail.
As previously mentioned, the controller card is striping the data as well as performing the XOR function on that data as well—the amount of logical computations the controller is doing every second is staggering. Today’s RAID controllers are intricate pieces of hardware, including specially designed processors and SDRAM memory banks to provide performance and redundancy.
RAID Introduction
Storage systems preserve data that has been processed and data that is queued up to be processed and have become an integral part of the computer system. Storage systems have advanced just as other computer components over the years. The RAID storage system was introduced over 15 years ago and has provided an excellent mass storage solution for enterprise systems. Let’s get a little more history about the RAID concept and they work.
Common RAID Configurations —the pictures below graphically show how RAID Arrays are put together (this is handled by the RAID configuration.) Follow the letters to see how the data stripes jump between drives.
RAID Recovery
RAID storage systems are designed to deal with failure. While hardware failure is a strong reason why some RAIDs may fail, there can also be other failures that make the data inaccessible. If your client is having problems with their RAID Array, then Ontrack Data Recovery is your solution.
A RAID recovery evaluation is really the combination of two very important steps. First is the array rebuilding and this has the potential of taking the most time. This investment in time is required in determining the original configuration and getting a quality recovery. The second step is to work on the logical file system. Today’s enterprise journaling file systems are highly complex; if the RAID Array is out of order there will be thousands of errors within the file system and files will be corrupted.
Some of the design goals of the RAID storage system were to provide performance improvements, storage reliability and recovery, and scalability. The redundancy concept employed in the RAID system is unique and provides a method to recover if one drive should fail within the system. In fact, today’s RAID controller cards have the ability to continue reading and writing data even if one drive is ‘off-line.’
Pandora Recovery is a powerful free tool that provides its users an effective way to attempt recovery of permanently deleted files. And that does not mean restoration of a file from Recycle Bin. Pandora Recovery actually recovers files permanently removed from Recycle Bin, files originally deleted using Shift + Delete keys bypassing Recycle Bin and files deleted from DOS prompt.
Pandora Recovery is not a backup tool – it can recover files that have been deleted months before Pandora Recovery was installed. And while the likelihood of successful recovery is negatively affected by the time passed since the deletion of files Pandora Recovery users were able to recover files eleven years after deletion and more than five years after re-formatting a hard drive!
Features:
1. Browse, Search, Preview and Recover deleted files
Pandora Recovery allows you to find and recover recoverable deleted files from NTFS and FAT-formatted volumes. Pandora Recovery will scan your hard drive and build an index of existing and deleted files and directories (folders) on any logical drive of your computer with supported file format. Once the scanning is complete you have full control over which files to recover and what destination to recover them to. You can BROWSE the hierarchy of existing and deleted files, or you can use SEARCH functionality to find a deleted file if you remember at least one of the following:
– full or partial file name,
– file size,
– file creation date, or
– file last accessed date
On top of that, Pandora Recovery allows you to preview deleted files of certain type (images and text files) without performing recovery. This feature becomes really important if you are forced to recover deleted files to the same drive. Currently you can preview files having several image file types (BMP, GIF, JPG, PNG, ICO,TIF, TGA, PCX, WBMP, WMF, JP2, J2K, JBG, JPC, PGX, PNM, RAS, CUR) and several text file types (TXT, LOG, INI, BAT, RTF, XML, CSS). Quick Viewer allows you preview file contents as text if it cannot find appropriate viewer for it. To use quick viewer you can select deleted file and or click the Quick Viewer icon or right click on deleted file and select ‘Quick View’. Quick View will then display a preview of deleted file.
Finally, Pandora Recovery allows you to recover deleted files, without any limitations on application’s end. Note that successful recovery might not be possible – if the original location has been reused by operating system to store different content the integrity of the original content has not been preserved and the recovered data will most likely be corrupted.
2. Surface (cluster) scan
Surface Scan – Embedded Image Preview
Click Image to Enlarge
As of version 1.1.20 Pandora Recovery implements surface (cluster) scan of targeted media, which allows:
– recovery of files whose MFT record has been reused by OS,
– Recovery of files from reformatted media,
– Recovery of files from discs with damaged or missing file allocation table.
Disk surface scan enumerates all unused clusters on the disk drive and searches for disk areas probably occupied by the data of removed files. Using this method of search, we do not use information contained in the file allocation table. So, it can work even on reformatted disk drives (except in) the case when low-level formatting was performed).
Almost each type of files has its unique features, for example the beginning of a JPEG file always contains JFIF magic word. Using this information we can determine what areas of the disk may contain deleted files data. Knowing the structure of a file (which is common within one file type) we can determine the length of the file and then recover file data.
File name and file attributes information are stored separately of the file data in file allocation table (FAT in FAT32 file system, MFT in NTFS), so we cannot determine the name of the file, its attributes, file path and recover data of file alternative data streams.
This method works only with non-fragmented files. Data of non- fragmented file(s) are stored in contiguous neighbor clusters, while data of a fragmented file may be stored in different places of the disk.
Remarks: In NTFS small files (less than 1KB in size) reside only in the Master File Table (MFT), so we cannot find such files using the Surface Scan Method.
3. Recover Archived, Hidden, Encrypted, Compressed files
Pandora Recovery can recover not only ‘regular’ files, but also archived, hidden, system, sparse, encrypted and compressed files.
Windows 2000 introduced Encrypting File System (EFS), which supports file encryption. EFS service runs on top of NTFS and encrypts or decrypts files or folders transparently for users and applications. Pandora Recovery does not decipher contents of encrypted files. Instead of that it copies the content of an encrypted file in raw mode just like data back-up applications do.
Encrypted and compressed files will be color-coded once drive indexing has been completed. Names of the encrypted files will be displayed in GREEN, while compressed files will be BLUE.
4. Recover Alternate Data Streams (ADS)
In NTFS all files have at least on file stream also called the unnamed $DATA stream. The unnamed stream contains the actual data of the file. Alternative streams usually contain file-related metadata. For example, MP3 music file may contain unnamed stream, where data are stored, and one or several additional streams where composition genre, album name and author’s name are stored.
While file might be usable without the original content of its Alternate Data Streams this useful information should be recovered. Alternative streams often contain file metadata, while file attributes contain information about how files are displayed in Windows Explorer.
5. Recover Images, Documents, Movies, or any other type of files
Pandora Recovery handles files regardless of their type, size or any other attribute. As long as the space on the logical drive has not been reused by operating system Pandora Recovery can successfully recover content of any deleted file.
If you want to search for unspecified deleted images or movies the best strategy is to search for common file extensions:
– *.avi, *.mpg, *.mov for video
– *.bmp, *.jpg, *.png, *.gif for images
6. Recovery success estimate
When a file has been deleted, the disk space occupied by the file can be reused by file system. The file system can reallocate this space for data of newly created files. Once operating system does that the deleted file becomes partially or completely overwritten. There are more chances that an overwritten file is corrupted and can not be recovered successfully. Overwritten files can still be recovered, but they likely will not be usable.
If clusters once used by a file have been reused by another already deleted files Pandora Recovery will display 0% as ‘overwritten’ value, which means the clusters are not currently in use. Still, the recovered data is likely to be corrupted.
Pandora Recovery gives you an estimate of recovery success by displaying the percent of clusters reused by operating system. To view the estimate move the mouse pointer over a file for a second or two until popup tip is displayed. The tip will say “Overwritten: 50%” or “Overwritten: 0%” . The higher the percent, the lower the chance of successful recovery.
Names of files with partially or completely overwritten clusters are displayed in RED.
7. Review File properties and Drive properties
Pandora Recovery displays properties of files: file type, deletion status, percent of clusters reused (‘overwritten’ attribute), path and size of the file, date created and date modified and common attributes (compressed, archived, hidden, system, encrypted…). To display file properties you should right-click on a file and select ‘Properties’ from context menu.
Pandora Recovery also displays properties of available logical drives: Drive letter, volume label, type, file system, total and available size. To display drive properties you should right-click on a logical drive and select ‘Properties’ from context menu.
8. Recover to Local Hard Drive, Network Drive, or Flash Drive
Pandora Recovery can use local hard drive, remote network drive or flash drive as a recovery destination folder. Pandora Recovery even allows you to recover deleted files on the same drive that the deleted files resided on originally.
IMPORTANT! To increase the success rate of file recovery it is strongly recommended that you recover your deleted file to a secondary hard drive, a network drive, USB Flash drive, or other external media. While recovery to the same drive that the deleted files reside on is physically possible it may lead to partial or complete loss of your deleted content. Non-deleted files will never be put at risk in either circumstance.
If your lost data resides on your C: it is recommended you perform your file recovery by physically removing the C: drive from your computer and attaching it as a slave on another computer and then performing the file recovery using that other computer.
9. Recognizes FAT16, FAT32, NTFS, NTFS5 and NTFS/EFS
NTFS (New Technology File System) is the default file system in all modern operating systems from Microsoft, such as Windows NT, Windows 2000, Windows XP and Windows Vista. NTFS 5.0 is improved version of NTFS introduced with Windows 2000.
NTFS replaced Microsoft’s previous FAT file system, used in MS-DOS and early versions of Windows. NTFS has several improvements over FAT such as improved support for metadata and the use of advanced data structures to improve performance, reliability, and disk space utilization plus additional extensions such as security access control lists and file system journaling. The exact specification is a trade secret, although (since NTFS v3.00) it can be licensed commercially from Microsoft through their Intellectual Property Licensing program.
NTFS5 (where 5 comes from internal enumeration of Windows 2000, the operating system which first implemented NTFS v.3.00) includes several new features over its predecessors: disk usage quotas, sparse file support, reparse points, distributed link tracking and file-level encryption, also known as the Encrypting File System (EFS).
FAT16 based on the usual 512 byte sector size and used with MS-DOS and Windows 3.xx, has a maximum partition size of 2 Gigabytes. FAT16 does not natively support alternate data streams or file permissions and it is not a journaling file system.
FAT32 was introduced with Windows 95. It is the standard file system used in Windows 95/98/Me. The theoretical Maximum partition size for the FAT32 file system is 8 Terabytes. However, in Windows 95 and 98 the size is limited to 127.53GB because to the disk 16-bit disk utilities included with the operating systems. Windows 2000 and later operating systems should be able to support the theoretical maximum size of FAT32, but Microsoft has placed an artificial limit which only allows formatting up to 32GB volumes on Windows 2000 and XP. The Maximum file size that can exist on a FAT32 formatted partition is 4GB. This has become a major factor in rendering the FAT32 file system obsolete for use on a personal computer hard drive. Media files can easily exceed this maximum size. FAT32 does not natively support alternate data streams or file permissions and it is not a journaling file system.
10. Compatible with Windows Vista, Windows XP, Windows 2003 and Windows 2000
Pandora Recovery requires Windows 2000, Windows XP, Windows 2003 Server or Windows Vista for installation. Pandora Recovery currently only recovers deleted data from NTFS formatted drives. There are no plans at this time to create a version compatible with the Mac, Linux, or any other non-Microsoft Operating System.
11. Wizards, hints and context-sensitive help
An easy way to get started in recovery your deleted files is by using the Wizard. By default the Wizard will open automatically when you start Pandora Recovery. You can also access the Wizard by clicking on the wizard icon in the program toolbar.
How does it work?
When you delete a file on FAT32 or NTFS file system, its content is not erased from disk but only reference to file data in File Allocation Table or Master File Table is marked as deleted. It means that you might be able to recover deleted files, or make it visible for file system again.
Search Deleted Files – Click Image to EnlargePandora Recovery allows you to find and recover recoverable deleted files from NTFS and FAT-formatted volumes, regardless of their type – you can recover pictures, songs, movies or documents. Pandora Recovery will scan your hard drive and build an index of existing and deleted files and directories (folders) on any logical drive of your computer with supported file format. Once the scanning is complete you have full control over which files to recover and what destination to recover them to. You can BROWSE the hierarchy of existing and deleted files, or you can use SEARCH functionality to find a deleted file if you remember at least one of the following:
– full or partial file name,
– file size,
– file creation date, or
– file last accessed date
On top of that, Pandora Recovery allows you to preview deleted files of certain type (images and text files) without performing recovery. This feature becomes really important if you are forced to recover deleted files to the same drive. Currently you can preview files having several image file types (BMP, GIF, JPG, PNG, ICO,TIF, TGA, PCX, WBMP, WMF, JP2, J2K, JBG, JPC, PGX, PNM, RAS, CUR) and several text file types (TXT, LOG, INI, BAT, RTF, XML, CSS). Quick Viewer allows you preview file contents as text if it cannot find appropriate viewer for it. To use quick viewer you can select deleted file and or click the Quick Viewer icon or right click on deleted file and select ‘Quick View’. Quick View will then display a preview of deleted file.
Pandora Recovery FREE DOWNLOAD
Download from Download.com – recommended
Download from local server (if the above location is not available)
http://www.pandorarecovery.com/
To install Pandora Recovery please do the following:
1. Download the Pandora Recovery installer from one of the download servers listed above.
2. Run the installer on the computer you wish to use to recover the deleted data with. After clicking the link above, simply click RUN or OPEN when the dialog window pops up.
3 . Proceed with the installation.
4 . Once installation is complete run the software and follow the wizard to start recovering deleted data.
IMPORTANT! To increase the success of file recovery it is strongly recommended that you recover your deleted file(s) to a secondary hard drive, a network drive, USB flash drive, or other external media. While recovery to the same drive that the deleted file(s) reside on is physically possible it may lead to partial or permanent loss of your deleted content. Non-deleted file(s) will never be put at risk in either circumstance.
If you deleted an important file from recycle bin that you really needed, do you have any way to get it back? Some people hit the fan and begin to drastically panic if the file they deleted was very important. In fact, you can recover deleted files from the Recycle Bin. The file you deleted from the recycle bin hasn’t actually been permanently erased yet, it is still hidden deep within your computer’s hard drive. All you need to do is know how to find it, and with the aid of new advanced software’s, finding and recovering that deleted file can be as simple as a few clicks of your mouse.
A common misconception is that once you delete a file from the Recycling Bin it has been erased forever, and it is no longer present on the Hard Drive. Luckily for you this is not true, you see when you click empty recycle bin, windows does not actually delete the files, Those commands just generally delete a file’s name from the directory so it won’t show up when the files are listed. But the information itself can live on until it is overwritten by new files. Therefore the file has not actually been deleted; instead the space that it occupied has been freed up and listed as re-use space. Now until that file is overwritten by new data your file is still there and can be retrieved.
The first thing you should do if you want to get your files back is to stop downloading or installing any new data on the hard drive. The reason being, that once you install new programs or download new data onto the Drive it could use up the space created by your deleted file and it will be a lot harder to recover. Having more available space on your hard drive will increase your chances of recovery, as Windows avoids using up space that has recently been freed; therefore if you have plenty of room on your hard drive it will use that before it touches the space from the deleted file. Although I still strongly recommend you act as soon as possible and do not download any new material to your hard drive, that way you will most certainly have your deleted files back.
Therefore in order to recover deleted files from the recycle bin you must obtain a good Undelete Software, these software’s can recover data from almost any data drive like hard disks, USB drives, Zip drives, SD Cards etc.. The software’s can also recover data from data cards in digital cameras and cell phones. The good news is that most Undelete Programs offer a Free Download, so you can try out their software at no cost to see if you are able to recover your deleted files from recycle bin.
The IT industry has become even more complex in the past few years with the advent of regulatory compliance requirements that all publicly traded companies in the US and other regions must adopt.
Perhaps your organization is already working through these requirements. If you are a consultant or non-publicly traded company, you may not be bound by these regulations – however your clients may be, so this information is critical for anyone in the IT industry.
Regulatory standards affect the broad areas of data privacy, security, retention, protection and accountability. Within these areas, checks and balances act to preserve the information and data. Investigative processes verify the integrity of privacy; security and data protection and audits are required for accountability.
The legal and business requirements protect a company from investigations or consequences but they also help safeguard consumer and patient information. Here’s a list of some of the common regulatory compliance laws.
This is by no means a comprehensive or industry specific list but serves as an example of the amount of data regulations that are already in place:
Data Regulations
Sarbanes-Oxley Act
Known as SOX, this Act requires company financial executives to be culpable for financial reporting. Independent auditors review financial controls and processes to ensure accurate financial reporting. Controls of records and processes are preserved to prevent fraudulent activities.
Healthcare Insurance Portability and Accountability Act
The Healthcare Insurance Portability and Accountability Act requires, among other things, the securing of patient information.
European Union Data Protection Directive
The European Union Data Protection Directive (EUDPD) standardizes the protection of data privacy for citizens throughout the European Union (EU) by providing baseline requirements that all member states must achieve through national implementing legislation
Payment Card Industry Data Security Standard
The four major credit card associations in the United States (Visa, MasterCard, American Express, and Discover Network) adopted a consolidated data security standard (Payment Card Industry Data Security Standard; PCIDSS). Compliance is required of merchants accepting these cards.
Japan’s Personal Information Protection Act
The Personal Information Protection Act. The Personal Information Protection Act applies to government or private entities that collect, handle, or use personal information of 5,000 or more individuals
Gramm-Leach-Bliley Act
The Gramm-Leach-Bliley Act addresses the protection of nonpublic personal information, requiring that financial records are properly secured, safeguarded, and eventually disposed of in a manner that completely destroys the information.
Breach Notification Legislation
California’s Senate Bill 1386 (SB1386) requires notification to California residents regarding any breach to the security of a computing system containing personal information.
Regulatory compliance issues can be really summed up by these simple items: “Keep it, Secure it, and Preserve it.” This can mean extra equipment and IT policies to maintain control over informationthat users may have previously horded on their machines.
One of the most important aspects to regulatory compliance is the 100% accessibility to the stored data. During data storage disasters, companies that require speed and quality turn to Professional Data Recovery Company for getting access back to regulatory data. In other situations, software that facilitates retrieving data is part of some IT department’s compliance process.
One of the least reported risks to electronic information is storage system failures. What happens when the server you have for compliance fails? How do you cope with a quarter-end financial audit when the business system database becomes corrupt? Who do you turn to when your company is in the middle of an SEC investigation and the electronic message server goes offline? These types of situations happen to corporations everyday. To help minimize this risk, several risk mitigation policies that storage administrators can adopt are outlined below:
Offline Storage System — Avoid forcing an array or drive back on-line. There is usually a valid reason for a controller card to disable a drive or array, forcing an array back online may expose the volume to file system corruption.
Rebuilding a Failed Drive — when rebuilding a single failed drive, it is import to allow the controller card to finish the process. If a second drive should fail or go off-line during this process, stop and get professional data recovery services involved. During a rebuild, replacing a second failed drive may change the data on the other drives.
During an Outage – If the problem escalates up to the OEM technical support, always ask “Is the data integrity at risk?” or, “Will this damage my data in any way?” If the technician says that there may be a risk to the data, then stop and get professional data recovery services involved.
Doing the Recovery Yourself – Some IT departments may have staff that has worked with automated data recovery or hard disk storage utilities. Depending on the cause of the data loss these tools could actually limit recovery efforts because the drive is experiencing intermediate failures. Some utilities on the internet are ‘free’ and promise to fix dead hard drives. Verify the source of the software and make sure that it comes from a reputable company that has a standardized development and quality assurance (Q/A) process. Untested software can yield unpredictable results.
When user desktop or laptop computer storage systems fail, do not assume that that their files are backed up, or synchronized, on the file server. At the same time, never assume that the data is completely gone.
Today marks the 50th anniversary of hard drive storage. When IBM delivered its first hard drive on September 13th, 1956, few could have imagined the impact it would have on our everyday lives. The RAMAC (also known as ‘Random Access Method of Accounting and Control’) was the size of two refrigerators and weighed a ton. It required a separate air compressor to protect the heads, had pizza-sized platters and was able to store a then whopping 5 megabytes of data. Now you can do all that with a mere pocket drive! What’s more – the RAMAC was available to lease for $35,000 USD, the equivalent of $254,275 in today’s dollars.
25 years later, the first hard drive for personal computers was invented. Using the MFM encoding method, it held a 40MB capacity and 625 KBps data transfer rate. A later version of the ST506 interface switched to the RLL encoding method, allowing for increased storage capacity and processing speed.
IBM made technological history on August 12, 1981, with the launch of their first personal computer – the IBM 5150. At a cost of $1,565, the 5150 had just 16K of memory- just enough for a small amount of emails. It’s difficult to conceive that as recently as the late 1980s 100MB of hard disk space was considered ample. In today’s era, this would be totally insufficient, hardly enough to install the operating system, not to mention a large application such as Microsoft Office.
When asked about the limitations of the early PC, Tom Standage, the Economist magazine’s business editor says: “It’s hard to imagine what people used to do with computers in those days because by modern standards they really couldn’t do anything.”
As a result of these major breakthroughs, the industry has grown from several thousand disk drives per year in the 1950s to over 260 million drives per year in 2003. During this period, the cost of magnetic disk storage has decreased from $2,057 per megabyte in the 1960s to $.005 today.
The future is bright
At present, the standard 3.5 inch desktop drive can store up to 750 gigabytes (GB) in data. But disk drives are set to become even smaller, more powerful and less costly. According to Bill Healy, an executive at Hitachi, drives containing hundreds of gigabytes will be small enough to wear as jewelry. “You’ll have with you every album and tune you’ve ever bought, every picture you’ve ever taken, every tax record.”
Having five disk drives in your household is becoming increasingly commonplace: PCs, laptops, game systems, TiVo® video recorders, iPod® – just to mention a few. Experts believe that someday households will have up to 15 disk drives, some of which may appear in your TV set, cell phone or car.
In fact, the industry is expected to deliver as many drives in the next five years as it did in the last 50 years. Industry analysts such as Gartner, IDC and TrendFOCUS believe that the global hard drive market will continue to experience impressive unit and revenue growth.
Take the good with the bad
As new devices hit the market, and the amount of stored data escalates the potential for data loss is greater than ever. No matter how strict your back-up policy or how heavily you invest in data protection – somewhere along the line data loss will occur.
Electronic data storage needs continue to grow. As your client’s organization produces more information in electronic format, storage space is becoming increasingly important.
Managing data storage for performance, integrity, and scalability is the next summit in Information Technology management and planning.
It wasn’t long ago that having a single volume in the terabyte size range was rare for extremely large organizations. With the advent of IDE RAID and SATA RAID capabilities, large storage systems are within reach of medium to small businesses.
Let’s put things into perspective – how much space is 1TB?
Number of Bytes | What that relates to |
1 Byte | One character (letter or number) |
1KB (Kilobyte) 1000 bytes | 3 or 4 typed manuscript style pages |
1MB (Megabyte) 1,000,000 bytes | Average size of a novel (300-400pgs); 1 diskette |
1GB (Gigabyte) 1,000,000,000 bytes | Approximately 20 sets of encyclopedias |
1TB (Terabyte) 1,000,000,000,000 bytes | A small library (approx. 5,000 books) |
Number of Bytes What that relates to
To get the best performance and reliability from any storage space, strategic storage planning is essential. This month’s technical article will review the importance of the file system and planning considerations.
The File System’s Role
The file system’s role is a layer above the storage device(s) itself. The file system manages the individual allocation units of the volume and provides hierarchical organization for the files. Managing the allocation units of the files requires algorithms that will know where to write file data and have a method of verifying that the data was written correctly.
Hierarchical organization is the logical formation of directories and underlying structures. For instance, a storage volume that has millions of files on it will have specific data that describes the directory or folder structure of where these files belong. This directory or folder structure has integrity checks and balances to ensure that the indices reliably point to the user data.
Today’s file systems track more than just the name of the file or directory structure. Additional information called Metadata is also stored. Metadata is data about data. Essentially, the file system is saving more details about your files and is storing this along with attributes of the file. Some file systems record only the minimum of metadata (file name, size, time and date, start address), while other file systems record more information (file name, size, multiple time and dates, security details such as Read/Write/Execute/Delete privileges).
Some file systems are designed for specific hardware and storage media. For instance, the file systems used for CD-ROMs are quite different than those for floppy diskettes. Forcing these file systems on other media may be possible, but not practical. So while specific storage media, such as CD-ROM, DVD-ROM, magnetic-optical disks, and tape, have unique file systems, hard disk and hard disk storage systems can work with many different file systems.
Understanding these extra features of file systems will help in choosing the best one for the needs of the volume.
File System Considerations
During server planning, more time and research is spent on hardware, data space requirements, and application specifications than on how the data will be stored. The file system can become a low priority during the planning stages of a file or data server because the file system is inherent to the operating system. Sometimes it is assumed that this is best fit. However, your storage requirements may call for a more robust method of data organization on the hard disk(s). Investigate whether the operating system you are planning to use allows the other file systems to be used.
If you have a choice of file systems, here are some requirements to consider:
• Volume Size
• Estimated number of files on the volume
• Estimated size of files on the volume
• Shared volume requirements
• Backup Requirements
Volume Size
Volume size is an important place to start for planning. However, this is only the start since strategic planning involves scalability—can it grow as the need arises without interruption of service to the users? The axiom of filling free space is all too true for data volumes. It is not uncommon to add a terabyte of storage and in six months it’s already half full.
Two terabytes (2TB) has become the initial hurdle for many file systems. This limit starts with the SCSI command set being limited to 32-bit logical block addressing. Therefore, a single SCSI LUN using 512 byte block size cannot access over 2TB. File systems that have been used on these systems have been ‘adjusted’ to handle extremely large volumes. However, volumes that are nearing the 2TB limit may be stressing the limits of the file system.
Estimated Number of Files on the Volume
The next item to plan for is the number of files that could potentially be stored on the volume. Earlier we discussed Metadata and how the file system uses this to describe the files that are stored. This means there is going to be a certain amount of volume space used by the file system just to manage the files that are there.
File systems that are not built for excessively large directories will slow down applications that access them. This can adversely affect users that have thousands of files on a volume that has millions of files.
Estimated Size of Files on the Volume
The next consideration is the sizes of the files that will be on the volume. Organizations that are running large database servers usually have the need to be able to pre-allocate very large files in the gigabyte range of sizes. The file system and operating system need to be able to handle this level of input and output. For these types of enterprises’ systems, expectations are high for performance and integrity. Will the file system be able to handle those extremely large files?
Shared Volume Requirements
There are mixed environments in many organizations today. Some organizations may have three or four different platforms of computer systems; from mainframe systems to 64-bit Sun machines, from Apple desktops to Intel based machines. Some of these systems may share storage space. Will the volume support mixed data types? Additionally, will the operating system that manages the file system allow for different types of data streams to be accessed simultaneously?
Backup Requirements
Large volumes present a challenge for backup procedures. Due to the amount of data, restorations can take days. There are some file systems that have ‘Snap-shot’ technology incorporated into the backup software. This technology saves critical file system metadata. This, along with incremental file backups, is part of entire system scheme of data archiving.
These considerations should be matched with hardware specifications to get the best performance, integrity, and growth capability.
Data Recovery is always an option for these types of devices. The quality of the recovery depends on how much usable data there is. In the British television program mentioned earlier, after the flash drive was shot out of the cannon and damaged, the producers sent the damaged flash device it to professional data recovery company After working with the device and its pieces, the engineers were able to recover the data on the device and found the data the producers were expecting.
Never assume that the data is gone when physical damage has occurred. The experienced data recovery engineers are capable of repairing complex electronics on USB flash drives.
What about deleted or reformatted USB flash drives? Similar to hard disks, when a USB flash drive is reformatted or data is deleted, the file system addresses to the data are erased—not the data itself. Even if some files are re-saved back to the device, there may be a chance that the information is recoverable. In simple deleted recovery situations, do-it-yourself solutions by using the professional software would be able to find the data and bring it back. In more complex situations where data has been restored back to the device, a trained data recovery engineer would be able to tell the difference between the newly written data and the original data. After an evaluation, the user would know exactly which files sustained damaged and which ones did not.
As long as the flash media is not physically damaged, a quick recovery choice for USB flash drives is that chooses the Remote Data Recovery service from professional data recovery company. The remote engineers can work on your flash drive while it’s still plugged into your computer and has access to the internet or to a modem.
As far as reliability goes, USB flash drives are very durable. They are “hot-swappable” (that is, removable without shutting down the computer) and “solid-state” (that is, no moving parts). They’re great for transferring data between computers. A British television program (“The Gadget Show”) decided to put flash drives to the test. They ran over them with a car, blasted them out of cannon, and baked them in a soufflé at 400° F! What was the result? The flash drives shot out the cannon suffered because they were broken into little pieces; the rest worked just fine and retained their data.
For the majority of USB flash drive users, their drives will never go through that type of punishment. It seems that the biggest risk in using these devices is simply losing them! They are so small and compact that it would be easy to misplace a USB flash drive. Most of them come with neck strap or keychain clip that allow them to be with you constantly.
Most industry sites define flash drives as a compact storage and transporting device whereas most dictionaries define flash memory as a computer chip with a read-only memory that can be electronically erased and reprogrammed without being removed from the circuit board. By definition, using a flash drive as an active storage area could pose a risk. For instance, one user used a flash drive like a second document folder. The user was creating and editing documents on the device with their word processor re-saving active documents every five minutes. This constant writing wore out the flash memory. Just like EEPROM chips, flash devices have a lifespan (this depends on the number of write cycles, check with the manufacturer find out the expectancy rates of your particular model), however, there is no limit to the number of times data can be read.
Security is the final risk. A common use for a flash drive is to transfer files from work to home. If the flash drive was lost or stolen during the transport, proprietary company information would be compromised. In fact, most small to large companies have strict policies of what types of information can leave the premises. This highlights the importance of data encryption.
There are a number of software encryption products that will maintain data security even if the flash device falls into the wrong hands. In fact, most USB flash drives come with some sort of free encryption software; however the free software may not meet your data protection requirements. If you use your flash drive for your company’s information or for your own personal information, be sure to purchase quality encryption software. The manufacturer of the flash device should have a recommendation of software on their Web site.
