Generate Large Files in Linux using dd

The programming community often requires large files for stress testing programmes. For example, sometimes such files are required to check the response time of certain programme or testing request handling capacity of servers. Many times, it does not matter what the contents of the file are, however, it is often difficult to find such large files when the need arises. Linux provides us a fast, efficient way of generating such huge files through simple command line options.

For the purposes of this demo, I am using Ubuntu 16.04 and 18.04 linux distros. Ubuntu provides dd command to create such huge files in a matter of seconds. A typical dd command to create a 1 GB file is given below.

dd if=/dev/zero of=big_file.txt count=1024 bs=1048576

parameters of dd:

if - input file from which the content is read
of - output file where the content is written
count - number of blocks in output file
bs - number of bytes in each block

The above command creates a file named big_file.txt of size (count*bs) i.e. 1024*1048576 bytes (1 KB * 1 MB = 1 GB) and fills it with stream of null characters taken from input file /dev/zero.

As you can see above, a file of size 1 GB has been created. The data transfer rate of 1.2 GB/s is visible.

Let us check how much time it actually takes to create a 2 GB file.

The above execution shows that it barely takes approximately 4 seconds to create a file of size 2 GB.

For most of the requirements, the above specified dd commands and its variations will suffice in generating large files for your requirements.

References:

https://linux.die.net/man/1/dd

Comments

Programmer's Guide: The "set" command in bash

The "set" command in shell allows you to set various shell options and positional parameters. The purpose of this post is to highlight how much this command has been helpful to me in my bash programming. I would not go into much depth about all the possible flag, because you can find many posts and documentation on the internet, like this post by GNU, which cover all the possible options that the "set" command provides. I will mainly focus on two options, The " set -x " option: This option allows us to print commands and their arguments, after they are expanded (interpolated) but before they are executed. Let us execute a simple hello world programme in bash. When the programme is executed, we see the entire printf statement being printed before the actual "Hello World!" message is printed. This way, we can check the command that is being executed along with its output. Setting the flag also expands any variables or ar...

Network scanning with nmap

nmap or network mapper is a great open-source tool for network scanning and port discovery. The detailed description of nmap is available on its official website . An interesting fact about nmap and its wide ranging applications from the website :p "...It was even featured in twelve movies , including The Matrix Reloaded , Die Hard 4 , Girl With the Dragon Tattoo , and The Bourne Ultimatum ." Hoping that this has generated enough curiosity in you, lets focus on the very basic use of nmap in network subnet scanning. Network scanning is useful to detect hosts in the network that are reachable from the server on which nmap is run. It is also useful in security auditing of the servers exposed to internet. We have a setup of 4 VMs on a local network with each having an IP address. The local network subnet is defined as 10.0.1.1/24 and the IPs in the below images verify this fact. The goal is to verify that nmap is able to detect all the hosts in the local subnet 10...

True or False with Bash

One of the first rules that is taught in any programming course is that 0 means False and anything that is non-zero is considered True. But when you come to system programming in Unix environment, you need to completely forget this basic programming rule. In unix shells, this rule is flipped and 0 means true / success while anything non-zero means false / failure. Every executed command in unix shells returns an exit status according to this convention. Let's see some examples in bash We will use the echo $? to get the exit status for the previously executed command. (The detailed explanation of this command is out of the scope of this article) The first example is as always to print "Hello World!" There are two ways of printing "Hello World!" as shown above. In both the cases, the exit status is 0 as the printing is successful. Let's check the true and false variables. As you can see, false returns 1. Another example using conditionals. ...

Shodh for Truth

Search This Blog