Posts Tagged Linux

How to switch Google Calendar from agenda view to week view in Thunderbird

I recently installed the Google Calendar add on and I also noticed that it would only display the agenda view (as other reviews have previously stated here). I fixed this (on Linux) by editing “~/.thunderbird/locymkyh.default/session.json” and changing:

"tabURI":"https://www.google.com/calendar/m","clickHandler"

to

"tabURI":"https://www.google.com/calendar/render??#g%7Cweek-2+22889+22895+22892","clickHandler"

NOTE: The ‘/locymkyh.default/‘ part of the path is uniquely generated for each user profile so yours will be different. Use the following command in a terminal to see yours:

find ~/.thunderbird -type d -iname '*.default'

Also, I don’t know what “week-2+22889+22895+22892” means to Google, but I don’t think it matters. This change should at least get Google Calendar to switch from the mobile agenda view to the desktop week view. If you have problems then try removing “-2+22889+22895+22892“. With and without the numbers worked for me.

, , , ,

1 Comment

How to read all lines of a file into a bash array

This blog post has received more hits than I had anticipated. It’s enough that I decided to revise it to improve the quality of the code (that people appear to be using). The original code examples were specifically written to explain the effects of IFS on bash parsing. The code was not intended to be explicitly used as much as it was to illustrate a point. It was also created in a proprietary embedded environment with limited shell capabilities, which made it archaic. The original post follows this update. Read it if you’re interested in IFS and bash word splitting and line parsing. Click here for a thorough lesson about bash and using arrays in bash.

There are two primary ways that I typically read files into bash arrays:

Method 1: A while loop

The way I usually read files into an array is with a while loop because I nearly always need to parse the line(s) before populating the array. My typical pattern is:

declare -a myarray
let i=0
while IFS=$'\n' read -r line_data; do
    # Parse “${line_data}” to produce content 
    # that will be stored in the array.
    # (Assume content is stored in a variable 
    # named 'array_element'.)
    # ...
    myarray[i]="${array_element}" # Populate array.
    ((++i))
done < pathname_of_file_to_read

Here’s a trivial example:

#!/bin/bash
declare -a myarray

# Load file into array.
let i=0
while IFS=$'\n' read -r line_data; do
    myarray[i]="${line_data}"
    ((++i))
done < ~/.bashrc

# Explicitly report array content.
let i=0
while (( ${#myarray[@]} > i )); do
    printf "${myarray[i++]}\n"
done

Method 2: mapfile aka readarray

The most efficient (and simplest) way to read all lines of file into an array is with the ‘readarray’ built-in bash command. I use this when I want the lines to be copied verbatim into the array, which is useful when I don’t need to parse the lines before placing them into the array. Typical usage is:

declare -a myarray
readarray myarray < file_pathname # Include newline.
readarray -t myarray < file_pathname # Exclude newline.

Here’s a trivial example:

#!/bin/bash
declare -a myarray

# Load file into array.
readarray myarray < ~/.bashrc

# Explicitly report array content.
let i=0
while (( ${#myarray[@]} > i )); do
    printf "${myarray[i++]}\n"
done

There are several options for the readarray command. Type ‘man bash’ in your terminal and search for readarray by typing ‘/readarray’.


Original post

 

By default, the bash shell breaks up text into chunks by separating words between white space characters, which includes new line characters, tabs, and spaces.  This action is called parsing.  You can control how bash breaks up text by setting the value of the bash built in “IFS” variable (IFS is an acronym for Internal Field Separator).  Bash will use each individual character in the IFS variable to decide when to break up text rather than using all characters as a whole.  So, for example, setting IFS to space and tab and new line, i.e. ‘ \t\n’, will cause bash to break up text every time it finds any combination of those three characters – not just when it finds the one combination of space followed by tab followed by new line.

Setting the value of a bash built in variable requires a different syntax than setting the value of a regular (non built in) variable.  The right hand side of the assignment must be prefixed with the ‘$‘ character.  Here is how to set IFS to the new line character, which causes bash to break up text only on line boundaries:

IFS=$’\n’

And here is a simple bash script that will load all lines from a file into a bash array and then print each line stored in the array:

# Load text file lines into a bash array.
OLD_IFS=$IFS
IFS=$'\n'
lines_ary=( $(cat "./text_file.txt") )
IFS=$OLD_IFS

# Print each line in the array.
for idx in $(seq 0 $((${#lines_ary[@]} – 1))); do
line=”${lines_ary[$idx]}”
printf “${line}\n”
done

While the code above works fine, it is not very efficient to store a text file in a bash array.  Programmers new to bash often want to do this and aren’t aware that it isn’t necessary.  An alternative solution is to simply parse on the fly so no array is required, like so:

# Load text file lines into a bash array.
OLD_IFS=$IFS
IFS=$'\n'
for line in $(cat "./text_file.txt"); do
printf "${line}\n"
done
IFS=$OLD_IFS

If you need to keep track of line numbers, just count lines as you parse them:

# Load text file lines into a bash array.
OLD_IFS=$IFS
IFS=$'\n'
let line_counter=0
for line in $(cat "./text_file.txt"); do
let line_counter=$(($line_counter+1))
printf "${line_counter}: ${line}\n"
done
IFS=$OLD_IFS

Be aware that changing IFS in the scripts shown above only affects IFS within the context of those scripts. If you want to change IFS in the context of your running bash shell and all sub-shells (and other child processes) that it spawns then you will need to export it like this:

IFS=$'\n'
export IFS

– or –
export IFS=$'\n'

And if you want this change to be system wide (not recommended) then you need to put this into /etc/environment or /etc/profile, or whatever is appropriate for your system configuration.

References

http://mywiki.wooledge.org/BashFAQ/001
http://mywiki.wooledge.org/BashFAQ/005
http://tldp.org/LDP/abs/html/internalvariables.html
http://www.gnu.org/software/bash/manual/bashref.html#Word-Splitting

Disclaimer:

Please keep in mind that the references listed above know WAY MORE than me. As of this post I’ve only been bash scripting for about three months and I only do it on occasion – like maybe once every three weeks – to solve some IT or embedded development issue. My posts are only meant to provide quick [and sometimes dirty] solutions to others in situations similar to mine. If you really want to be good at bash scripting then spend some quality time with the official documentation or a good book.

, , , , , , , , , , ,

15 Comments

How to fill unused drive space with zeros in Linux

Filling unused drive space with zeros is useful for cleaning a drive and for preparing a drive image for compression.  A drive whose unused space is filled with zeros will compress to a much smaller size than one that is not.  There are two methods typically used to fill a Linux drive with zeros:

A simple shell script

The following shell script simply creates a file and fills it with zeros until all drive space in exhausted, then it deletes the file.  This isn’t a perfect solution, but it’s better than nothing.

#!/bin/bash
cat /dev/zero > /zero.fill
sync
sleep 5
sync
rm -f /zero.fill

The ‘zerofree’ utility.

The zerofree utility, written by Ron Yorston, only works on devices containing an ext2 or ext3 file system. It seems to work fairly well, however I have crashed one system with it so be careful. You should probably clone the drive you plan to zerofree before doing so.

References

http://intgat.tigress.co.uk/rmy/uml/sparsify.html
http://packages.ubuntu.com/search?keywords=zerofree

, , , , , , , ,

1 Comment

How to clone a drive over the network in Linux

In my embedded work I often need to clone disk drives, USB memory sticks, and miscellaneous other types of mass storage devices.  Here is a simple and easy way to clone/copy the primary hard disk of a source machine onto the primary hard disk of a destination without taking anything apart or using tertiary storage.  Of course, you may need to update drivers and system configuration on the destination machine after copying the source machine to it (if the hardware in each machine is not identical).

Assuming that your destination machine disk drive is as large as your source machine disk drive (or larger), you can copy the contents of the source machine drive directly to the target machine using the netcat (‘nc’) application. Since netcat uses the network, you may encounter problems if you’re running a firewall on either machine. If you are then either temporarily disable them or reconfigure them to allow traffic on whichever port you tell netcat to use. I typically use port 12345. For example, I needed to copy/clone the contents of the primary hard disk so I booted each machine with a live Linux CD and executed the following commands:

[Target Machine (assume IP address 192.168.0.10)]

# nc -l 12345 | gunzip - > /dev/sda

[Source Machine]

# cat /dev/sda | gzip - | nc 192.168.0.10 12345

And then I waited. This resulted in about 8MB/sec throughput (on average) over my 10MB/sec LAN, which is not bad. Saturation would be 10MB/sec so gzip was definitely improving performance since it was compressing at least 2MB/sec. It took a while to transfer, but I didn’t have to use a tertiary mass storage device and I didn’t have to worry about copying non-regular “files” such as device nodes and fifo’s/pipes, which would normally require the use of tar or rar. I just started the process and let it run while I did other work.  I suspect it ran for a couple of hours, but I wasn’t keeping track so I don’t know for sure.

This method of cloning a drive completely ignores drive geometry and file systems so it will not work on all machine configurations, but it’s likely to work for most and it’s an easy set and walk away type of solution to an otherwise complex problem.

After using this method to clone the source machine, I used the Linux LVM utilities to resize the partitions and file system on the destination machine so no space was wasted.  This can also be done to NTFS partitions using a Linux live CD.

, , , , ,

1 Comment

How I solved my nVidia driver kernel module API mismatch problem.

I recently ran into a problem with my proprietary nvidia video driver on Ubuntu Linux 9.04.  It seems that there was a conflict between the driver I downloaded directly from http://www.nvidia.com/Download/index.aspx?lang=en-us and the driver installed by Ubuntu’s restricted driver manger (found on the gnome System menu under Administration | Hardware Drivers).  The problem was that the kernel module API installed by Ubuntu did not match the driver installed by the installer I downloaded and the end result was that my screen was completely black/blank and no tty’s worked so pressing Ctrl+Alt+F[1-6] wouldn’t take me to a terminal.  (Executing dmesg at a command prompt would show messages indicating that there was an API mismatch.  This message would also be shown by running “startx” at the command prompt.)

Simply pressing the power button once was sufficient to get Ubuntu to gracefully shutdown the machine, thankfully.  After powering back up I selected the recovery item on the grub boot menu to boot into a root console.  If you don’t have a recovery menu item then add “single” (without the quotes) to the kernel command line in the grub menu and boot from there.  From the root console I did three things: I uninstalled the nvidia drivers that I manually installed earlier (that broke my system), I unloaded the nvidia kernel module, and then I manually searched for and removed all nvidia kernel modules.

Uninstalling the nvidia drivers I installed manually was easy.  All I had to do was run “nvidia-installer --uninstall”.  I got a waring because I was running in single user mode so I had to tell the installer to proceed anyway and it worked fine.  You will also get an error if X is running.  To terminate X you need to kill gdm by executing “/etc/init.d/gdm stop” as root, however it should not be running if you booted into the recovery console.

Unloading the nvidia kernel module was easy.  Simply execute “rmmod nvidia” as root.

Searching for and removing all nvidia kernel modules was tedious, but also easy.  This procedure is not the ideal solution, but I had no working video or networking and had to take relatively drastic measures.  First I had to find all the drivers.  I did this by executing “find / -type f 2> /dev/null | grep -i nvidia | grep -i \\.ko” as root.  This generated output similar to this:

/var/lib/dkms/nvidia/173.14.16/2.6.28-13-generic/i686/module/nvidia.ko
/var/lib/dkms/nvidia/173.14.16/build/nvidia.ko
/var/lib/dkms/nvidia/173.14.16/build/.nvidia.ko.cmd
...
/var/lib/dkms/nvidia/original_module/2.6.28-13-generic/i686/collisions/kernel/drivers/video/nvidia.ko
/var/lib/dkms/nvidia/original_module/2.6.28-13-generic/i686/nvidia.ko
...
/lib/modules/2.6.28-13-generic/kernel/drivers/video/nvidia.ko

Then I manually removed each kernel module by using the “rm -f <pathname_of_file>” command.

Finally I reran the nvidia installer, did a full install of the video driver, and then rebooted the machine by executing the “reboot” command as root.  When the machine finished booting I had video back and all was well in my world again.

, , , , , , , ,

7 Comments