Extract, Label, and Organize Data
In this IT Pro Challenge virtual lab, you will demonstrate your ability to use advanced Linux commands to manipulate files and data sets and to organize better and clean up your data. You will exercise your skills with Linux commands such as awk, grep, ls, tr, echo, and sort.
This challenge is ideal for learners who are familiar with the command line in Linux. It will help learners sharpen their skills in creating and managing files using the command line in Linux and managing data (such as: removing unwanted lines, rearranging and sorting data, displaying content, filtering on names, and translating letter cases). In this lab, you are an administrator on an Ubuntu Linux system. You will use advanced knowledge of the awk, sort, tr, ls, grep, and echo commands to organize, extract, and label data within the files on your system. You will organize, clean up, and output your data to a new and clean set of files.
Understand the scenario
You are a Linux system administrator. You need to manage data within your file system. To set the stage, you will use a series of Linux commands to make a directory, create files, and fill those files with various sets of data. For your first housekeeping task, you will clean up a file with excessive whitespace, using awk and the NF variable. Then, you will sort and rearrange data columns in an existing file to better organize the data, using awk and sort. Next, you will find and organize a list of files that have specific name requirements, using ls and wildcards. Finally, you will translate the contents of a file from lower-case to upper-case, using tr.
Create necessary files and directories
To start this lab, you are provided a series of common Linux commands that allow you to build out your working space quickly. The commands will build a file structure of relevant but unorganized and cluttered data. Your job will be to clean up and organize the data and these files. You will use the mkdir, touch, and echo commands to build the file and folder structure and populate your files with various sets of data.
Remove unwanted lines from a file
Your first task is to clean up a file that has excessive whitespace and incomplete data sets. You will use the awk command with the NF variable to condense data down, minimizing whitespace, and to find fields with missing data, so you can also remove the incomplete data sets. The cleaned-up output will be saved to a new file.
Rearrange the columns of an existing file
In this task, you will use awk and sort to re-organize a subset of data columns and ensure it is sorted appropriately. The newly sorted subset of data will be saved to a new file.
Select file names by using wildcards
This section exercises your skill with the ls command and using wildcards. You have a directory of files, and you need to create a list of some of the file names in the directory. You will identify the files of interest, create an ls command to list those file names using wildcards so that you only get output that includes the file names of interest. Then, you will save that output to a new file, so that you have a clean list of specific file names.
Translate the contents of a file to uppercase
For this final task, you have a data set where the content is not consistently formatted to your expectation. You will use the translate command (tr) to translate all characters to upper-case. The newly translated output will be saved to a new file.
Once you are confident that you have successfully completed the tasks in this challenge, you will submit your work to be automatically validated against the task requirements, to ensure that you have organized the data and files correctly.
In this hands-on challenge, you will extract, label, and organize data within a Linux file system. The skills exercised here are important for System Administrators, Security and Forensic Analysts, and Penetration Testers. The tasks you will perform are:
- Remove unwanted lines (excess whitespace) from a file and filter content by using the awk command.
- Rearrange and sort columns in a file by using the sort command.
- Select file names by using wildcards.
- Display file names by using the ls command.
- Convert contents of a file to uppercase by using the tr command.
- Redirect the output of a command to a new file.
Other Challenges in this series
- GUIDED CHALLENGE: Search Linux Manpages by Using the Command Line
- GUIDED CHALLENGE: Display Manpage Information by Using Sections
- ADVANCED CHALLENGE: Can You Use Several Manpages At Once?
- GUIDED CHALLENGE: Configure the Display of a File
- GUIDED CHALLENGE: Search for Files by Using Wildcards
- ADVANCED CHALLENGE: Can You Create Files by Using Parts of Other Files?
- GUIDED CHALLENGE: Search for Data by Using Grep
- GUIDED CHALLENGE: Sort Files
- ADVANCED CHALLENGE: Can You Use the Grep and Sort Commands Together?