NTFS and Linux – An Annoying Relationship

Very recently I wrote a blog about getting rid of FAT32 file systems because the maximum file size that it can handle is 4GB. Read it here if you are interested.

By the end of that blog I had decided that I would move to either ext4/NTFS and get rid of FAT totally. ext4 was only for linux and is not compatible with windows. NTFS on the other hand is compatible with windows and linux. I started off by copying stuff off my 16GB FAT32 pen drive and loading it on my laptop, to format it into an NTFS drive, so that I can load it all back again. I was done with this process in half an hour. Then I noticed something, all the files and folders in the drive had weird (read, write, execute) permissions now. Everything was a 777 (rwxrwxrwx). Meaning, every file had permissions to read, write and execute for user, group and others. That was an irritating sight to see on a terminal.

A little searching,  and I realized that, although NTFS has a much bigger file size limit, it does not retain permissions from a linux system. Uff! That was really disappointing. It was still possible to maintain permissions, but it was a long process and I was not going to complicate my life for basic backup and file transfer. I had to do some thinking and I realized that the best way to handle this was to use ext4 along with FAT32 and NTFS, in some fashion so I get the best of both worlds. So this is what I decided for the two 16GB pen drives and one 1TB hard disk I have for work.

  1. One of the 16GB pen drive would be turned into an ext4 system for use with Linux and also for those big files. Since most of the computers around me use linux, it would not be much of a problem and I do not want any pesky virus filled windows users to use my pen drive either.
  2. The other 16 GB pen drive will remain FAT. To be compatible across all machines. It would not be able to handle huge files but who’s handling huge files all the time.
  3. The 1TB hard disk would be split into two drives
    1. 800GB ext4 for use with my machine and other linux machines if required. Again to avoid virus laden windows users and their machines. (It is pretty simple to clean off viruses from a linux system later on though. But it needs some work. Maybe I will talk about it in some other post)
    2. 200GB NTFS for use with windows if required anytime in the future. A big portion, but I thought it might be required somehow.
Advertisements

Compress, Split and Join Tar files – Linux

Creating a tar file and splitting it for storing it in cds, dvds or any other storage with small capacity.

Create a tarball and compress it using bz2 or gz compression algorithms by using one of the following commands.

tar -cvjf tarofsomefolder.tar.bz2 /home/someone/Documents/*
tar -cvzf tarfileofsomeiso.tar.gz some.iso

To split you can use the split command
split -b 4500M tarofsomefolder.tar.bz2 "tarofsomefolder.tar.bz2.part"

You can use both of them together in a single command by piping the outputs of the tar command to the split command,

tar -cvzf downloads.tar.bz2 ~/Downloads/* | split -b 4500M - "downloads-part"

To extract these split tar files together, you have to cat those individual tar files either by listing them one after the other or by using a regular expression that covers all of it and then piping the output to a tar extract command. Something like,

cat downloads.tar.bz2.part* | tar -xvzf -

Mount a disk – Linux – cmd

To mount a disk (usually a file system that you usually go and click on using nautilus/caja (file browsers)) from the command line, use the following,

udisksctl mount -b /dev/disk/by-label/

Listing the content of /dev/disk/by-label/ will give you the names of the mountable disks/drives. Usually flash drives are mounted automatically so they do not have to be mounted. Unless you unmount it and want to mount it again. Use man udisksctl for more information. Also, I think it is udisks in some distributions.

Reaper on Linux, Wine – Blog – Part 1/2

Yesterday I was just going through the Reaper website and I noticed something really interesting. I was so happy to see it.

windows - wine

Seems reaper works well through wine. I have always thought of how the Reaper guys never ported it to Linux. Since Reaper feels so much like Linux, in philosophy and working.

So I did it. I installed Reaper on Fedora through Wine.

reaper-wine-linux

It runs reaaally well!. Super slick!. I love it. I haven’t done anything on it though. No recordings or projects yet. Will do it sometime in a few days. I will get back to you on how everything went. Hopefully it is not disappointing. 🙂

Screen Record, on Linux

This is a quick way to make a tutorial or how-to video for someone. On my fedora, I already had this command line app called, recordmydesktop and it works well.

All you have to do is head to the command line and say recordmydesktop test.ogv. This will record the screen and save it to a file called test.ogv. To end recording, you can hit Ctrl + c.  It takes some time to process the file and save it. You might see something like the image below.

record

There a lot of options, including ways to specify an area of the screen to record and things like adding a short delay before it starts recording. Check out the man page (man recordmydesktop) for more details.

This records in OGV format. But you can quite easily convert it to an mp4 file, if you’d like, with ffmpeg or some other conversion software. With ffmpeg, use the following command.

ffmpeg -i test.ogv -c:v libx264 -preset veryslow -crf 22 -c:a libmp3lame -qscale:a 2 -ac 2 -ar 44100 output.mp4

I’m not sure if this is default or it got installed along with some other package. Anyway, installation is as simple as, dnf install recordmydesktop, as a root user. Happy recording.

LF, CR, CRLF – Pressing Enter

Story (Skip it if you don’t have time)

I had a tough time once when i was processing text. I usually work on linux machines. I got this text file that was made on a windows machine and I was going to process it. The process involved adding a few characters at the start of every line. To do this. The usual thing to do is to replace ‘\n’ with ‘\nabc’ assuming abc are those characters to be placed at the beginning of every line.  This just did not work on this particular file that i received. It was only after i used a tool that displays every single character on the file that i realized that text made on windows machines add something called a carriage return ‘\r’, along with the line feed character ‘\n’. Thus the need for a post.

TL;DR Start reading from here..

So all this goes long back to the age of the typewriter. LF stands for Line Feed. It would just move the paper up by a line space and keep the same horizontal position. CR on the other hand stands for Carriage Return and moves to the start of the line on the same line.

So intuitively it seems that the windows people have got this right. Because CR and LF would move the pointer to the start of the next line. But is it necessary in the current age? That is another question. On linux, only the line feed character is used. On Mac, only the carriage return character is used.

The ASCII codes are given below,
CR = \r = ASCII code 13
LF = \n = ASCII code 10
CRLF = \r\n = ASCII code 13 and then ASCII code 10

Keep this in mind when you are processing text files originating from these operating systems.

Thanks for reading.

 

Temp Files – Bash Scripting

If you use bash scripts on a regular basis for file manipulations you would have come across times where the output had to be written to some temporary files. From which further handling can be done. This is quite common. But surprisingly after years, I have come to know the use of temporary files that the system itself safely creates in the /tmp directory. You do not have to worry about the naming or the location or such. You can use the mktemp command to create a temporary file which can be assigned to a file handler.

Lets look at a very common use. You have some text ready after some processing. You want to write it to a file. What you do before hand is say,
temp_file=$(mktemp)

This will create a tempfile and assign it to the file handler temp_file. Then you will probably want to write some stuff to this file. you can do this by just sending the outputs like this,
echo "my output" > $temp_file

Just using echo here for explanation purposes. You can continue appending your outputs of course, using >>.

You can use this file for further processing in a for loop or something just like you would do any other file. Like for example,

for line in `cat $temp_file`; do

This was one really useful tip because I do not have to worry about the file naming or location of the file or about removing the file later on.

Thanks for reading.