Generate Large Files in Linux using dd

The programming community often requires large files for stress testing programmes. For example, sometimes such files are required to check the response time of certain programme or testing request handling capacity of servers. Many times, it does not matter what the contents of the file are, however, it is often difficult to find such large files when the need arises. Linux provides us a fast, efficient way of generating such huge files through simple command line options.

For the purposes of this demo, I am using Ubuntu 16.04 and 18.04 linux distros. Ubuntu provides dd command to create such huge files in a matter of seconds. A typical dd command to create a 1 GB file is given below.

dd if=/dev/zero of=big_file.txt count=1024 bs=1048576

parameters of dd:

if - input file from which the content is read
of - output file where the content is written
count - number of blocks in output file
bs - number of bytes in each block

The above command creates a file named big_file.txt of size (count*bs) i.e. 1024*1048576 bytes (1 KB * 1 MB = 1 GB) and fills it with stream of null characters taken from input file /dev/zero.

As you can see above, a file of size 1 GB has been created. The data transfer rate of 1.2 GB/s is visible.

Let us check how much time it actually takes to create a 2 GB file.

The above execution shows that it barely takes approximately 4 seconds to create a file of size 2 GB.

For most of the requirements, the above specified dd commands and its variations will suffice in generating large files for your requirements.

References:

https://linux.die.net/man/1/dd

Comments

Using static IPs in Linux

It is often a requirement to assign static IP addresses to some important and permanent network interfaces. This is to avoid setting up a local DHCP server or relying on IP given by your network provider which may not be stable. This post shows steps to configure static IPs in Linux using Ubuntu 16.04 and Ubuntu 18.04 distros. We are considering two distros as network configuration has changed significantly in Ubuntu 18.04. There are two general steps to be followed: 1. Configure the network interface and assign a static IP. 2. Restart interface for the changes to take effect. Ubuntu 16.04 The current interfaces on the system are as follows: As can be seen, enp0s8 interface does not have any IP. Let's assign a static IP to it. In case you already have an interface with a DHCP assigned IP, you just need to change that IP and make it static. Edit the /etc/network/interfaces file and add the following code block. Prefer using the IP address range available for pri...

Simple HTTP Server in Python

Have you ever come across a need to share a file over a local network to a number of machines? Have you ever felt the need to share output of a task in text files available at a web-endpoint? Often, there is a requirement to quickly setup a light-weight web-server for such requirements. Python provides the easiest way to setup a simple HTTP server for such use cases with a single command. Python comes pre-installed with Linux as many of the linux libraries use python in background. Therefore there is no extra overhead of installing python. With support for python2 getting stopped in April 2020, many of the newer versions of linux distros come with default python3. Therefore this post will focus on setting up an HTTP server using both python2 and python3. We will be using Ubuntu 16.04, however it will work for majority of the linux systems where python comes pre-installed. Preparation Steps to follow before starting the server: Make sure you are in the directory which you want ...

Local domain name resolution using /etc/hosts

The /etc/hosts file is a powerful mechanism for managing the information about hosts in the local network in the absence of a local DNS server. Setup We have 4 VMs in the local /24 network. The following are the details: vm-1-ubuntu-16-04 - 10.0.1.11 vm-2-ubuntu-16-04 - 10.0.1.12 vm-1-ubuntu-18-04 - 10.0.1.21 vm-2-ubuntu-18-04 - 10.0.1.22 The VMs are reachable via their IP address but not by their hostnames. This is problematic because we have to remember their IP addresses everytime we want to access these hosts. It is would be much simpler to remember and access the servers by their hostnames. For that we need some sort of mapping between the IP addresses and their corresponding host names. There are 3 common ways of achieving this mapping. Setup a DNS server which handles resolution for your local network. Use an existing DNS server of the local Internet Service Provider (ISP) or any other higher level ISP. Note that a public static IP address is required for this s...

Shodh for Truth

Search This Blog