Generate Large Files in Linux using dd

The programming community often requires large files for stress testing programmes. For example, sometimes such files are required to check the response time of certain programme or testing request handling capacity of servers. Many times, it does not matter what the contents of the file are, however, it is often difficult to find such large files when the need arises. Linux provides us a fast, efficient way of generating such huge files through simple command line options.

For the purposes of this demo, I am using Ubuntu 16.04 and 18.04 linux distros. Ubuntu provides dd command to create such huge files in a matter of seconds. A typical dd command to create a 1 GB file is given below.

dd if=/dev/zero of=big_file.txt count=1024 bs=1048576

parameters of dd:

if - input file from which the content is read
of - output file where the content is written
count - number of blocks in output file
bs - number of bytes in each block

The above command creates a file named big_file.txt of size (count*bs) i.e. 1024*1048576 bytes (1 KB * 1 MB = 1 GB) and fills it with stream of null characters taken from input file /dev/zero.

As you can see above, a file of size 1 GB has been created. The data transfer rate of 1.2 GB/s is visible.

Let us check how much time it actually takes to create a 2 GB file.

The above execution shows that it barely takes approximately 4 seconds to create a file of size 2 GB.

For most of the requirements, the above specified dd commands and its variations will suffice in generating large files for your requirements.

References:

https://linux.die.net/man/1/dd

Comments

Local domain name resolution using /etc/hosts

The /etc/hosts file is a powerful mechanism for managing the information about hosts in the local network in the absence of a local DNS server. Setup We have 4 VMs in the local /24 network. The following are the details: vm-1-ubuntu-16-04 - 10.0.1.11 vm-2-ubuntu-16-04 - 10.0.1.12 vm-1-ubuntu-18-04 - 10.0.1.21 vm-2-ubuntu-18-04 - 10.0.1.22 The VMs are reachable via their IP address but not by their hostnames. This is problematic because we have to remember their IP addresses everytime we want to access these hosts. It is would be much simpler to remember and access the servers by their hostnames. For that we need some sort of mapping between the IP addresses and their corresponding host names. There are 3 common ways of achieving this mapping. Setup a DNS server which handles resolution for your local network. Use an existing DNS server of the local Internet Service Provider (ISP) or any other higher level ISP. Note that a public static IP address is required for this s...

Programmer's Guide: The "set" command in bash

The "set" command in shell allows you to set various shell options and positional parameters. The purpose of this post is to highlight how much this command has been helpful to me in my bash programming. I would not go into much depth about all the possible flag, because you can find many posts and documentation on the internet, like this post by GNU, which cover all the possible options that the "set" command provides. I will mainly focus on two options, The " set -x " option: This option allows us to print commands and their arguments, after they are expanded (interpolated) but before they are executed. Let us execute a simple hello world programme in bash. When the programme is executed, we see the entire printf statement being printed before the actual "Hello World!" message is printed. This way, we can check the command that is being executed along with its output. Setting the flag also expands any variables or ar...

True or False with Bash

One of the first rules that is taught in any programming course is that 0 means False and anything that is non-zero is considered True. But when you come to system programming in Unix environment, you need to completely forget this basic programming rule. In unix shells, this rule is flipped and 0 means true / success while anything non-zero means false / failure. Every executed command in unix shells returns an exit status according to this convention. Let's see some examples in bash We will use the echo $? to get the exit status for the previously executed command. (The detailed explanation of this command is out of the scope of this article) The first example is as always to print "Hello World!" There are two ways of printing "Hello World!" as shown above. In both the cases, the exit status is 0 as the printing is successful. Let's check the true and false variables. As you can see, false returns 1. Another example using conditionals. ...

Shodh for Truth

Search This Blog