Working with UNIX 'epoch'

An administrator friend was trying to find out on which date the last change of a password occured. He knew that this information was stored in the /etc/shadow file.

Well, the third field of the shadow file contains a number, which is equivalent to number of days from the epoch, which for UNIX was agreed to be January 1, 1970, at midnight, UTC.

So, let's see how we can convert this information into a meaningful date. For example, on my test machine I have:

$ grep root /etc/shadow
root:$1$.LbW0Bv2$keqd6WlAumjwvqRl2tu6U1:13117:0:120:7:::

The conversion is simpler that you may think - no complicated calculations, just yet another useful option of the date command:

$ date -d "1970-01-01 utc + 13117 days"
Wed Nov 30 00:00:00 GMT 2005

The above command means: show the date based on the given "string". The string used here says: use the epoch as the base date, and add to it the value of the 3rd field (for the user of interest) from the /etc/shadow file .

The unit at the end of the string is important! For example, consider the following:

$ date +%s
1137404710
$ date -d "1970/01/01 utc + 1137404710 sec"

Mon Jan 16 09:46:11 GMT 2006

In the above example we first convert the current date into the 'epoch' value, but expressed in seconds (notice that the +%s formatting is only available on the GNU-enhanced versions of date). We then used the date command to convert this number back into a proper date format.

You could use the output of the date command as part of the string used in the calculations. The following is for illustration purposes only, really. It's a long-winded way of finding the 'now' date.

$ date -d "1970-01-01 utc + $(date +%s) sec"

Mon Jan 16 10:02:54 GMT 2006

Finally, the same -d option can be used for creating any date stamp, for example:

$ date

Mon Jan 16 10:26:21 GMT 2006

$ date -d "+ 30 min"

Mon Jan 16 10:56:42 GMT 2006

$ date -d "+ 2 days"
Mon Jan 18 10:58:03 GMT 2006

xhost in RHEL4

I needed to illustrate how an operator could send an X application from a server to a workstation in his office (I'm talking here RHEL4 boxes).

Simple, done it loads of times - on the workstation do:

# xhost +server-name

On the server run the application from the command line and direct the output to the display on the workstation:

# xapplication -display workstation-name:0.0

WRONG! It's not that simple on RHEL4! Here, you need to enable TCP first, which is done as part of gdm (graghical desktop manager) configuration.

Run:

# gdmsetup

Select Security tab and notice that the following setting is "checked":

"Always disallow TCP connections to X server (disables all remote connections)"

Take the 'tick' off, restart the GUI (CTRL-ALT-Backspace), and all is well - the method shown above will now work.

I'm still about...

Anybody looking at this blog would be forgiven for thinking that I've given up...
No such luck. I'm still here, except that the project I've been involved in took me into a customer's environment where access to Internet is not easy.

How ironic is that! Internet has clearly become a victim of its own success. I still find it incomprehesible that one might be expected to work without access to e-mail and the web, yet most of large organisations don't seem to provide these facilities to visitor even or even contractors.

Time for 3G or similar?

Polish your Ingres in Polish

Been looking for a book on Ingres, hoping to find something on r3.
Funily enough, couldn't (and din't) find anything on amazon, etc. There are some excellent HOWTOs, notably at www.tldp.org, but nothing recent - most of the documents are several years old. Something more contemporary could prove quite useful. Now, there is a thought ;-)

However, found a book in a Polish on-line technical bookshop!
http://www.ksiegarnia-techniczna.com.pl/modules.php?name=Sklep&plik=lista&nazwa=opis&kod=837101144X

OK, it isn't exactly up-to-date, as was published in 1994, but I still found it amusing that one exists! And at all of 6.50 zl (about £1.00), I decided to order it!

Ingres 3 - part 2

Well, now that the software appears fully installed and configured, the daemons all hovering in the background waiting to spring to life, time to put some data in, and see if one can ever see it again ;-)

Logged in as user ingres, and off we go:

[ingres@fedora4 ~]$ createdb newone
Creating database 'newone' . . .

Creating DBMS System Catalogs . . .
Modifying DBMS System Catalogs . . .
Creating Standard Catalog Interface . . .
Creating Front-end System Catalogs . . .
Creation of database 'newone' completed successfully.

So far so good, now have a go at creating then selecting some data:

[ingres@fedora4 ~]$ sql newone
INGRES TERMINAL MONITOR Copyright 2005 Computer Associates Intl, Inc.
Ingres Linux Version II 3.0.2 (int.lnx/105) login
Fri Oct 21 09:49:07 2005

continue
* create table t1 (col1 char(20));
* insert into t1 values ('hello world');
* select * from t1
* \g
Executing . . .
(1 row)

┌────────────────────┐
│col1 │
├────────────────────┤
│hello world │
└────────────────────┘
(1 row)
continue
* \q
Your SQL statement(s) have been committed.
Ingres Version II 3.0.2 (int.lnx/105) logout
Fri Oct 21 09:52:46 2005

So, what's the conclusion? Well, I'm very impressed.

I suppose I will encounter some problems with the very installation I have just completed, as there are specific environmental settings, as well as kernel parameters that I did not investigate and adjust.

Another important issue a chose to totally ignore is that of user identity when performing the download and installation. This should also have been identified and used correctly. However, what the above exercise proved, is that Ingres for Linux is fine, and can be used 'out of box'.

Basic installation and configuration is simple - if the defaults are fine for you, that is!

Ingres 3 - part 1

A customer of ours is planning an upgrade to Ingres3, whilst changing from Dynix to Linux at the same time. So I thought that I'd refresh my acquaintance with Ingres, since I haven't seen this version yet. I have had some experience of looking after a Linux system running Ingres, but this is the first time I'll be installing r3 myself.

At this stage I'm not even sure that what I'm doing will work. This is going to be what I call a dry run - I'm going to play it by ear, following the installation steps intuitively, rather than following any particular set of instructions - so, the following steps should be taken more as a log of events, rather than a tutorial. I had a look at www.tldp.net , which does have an ingres HOWTO, but it is a really old one, and although most of the document might be OK, I decided not to risk it ;-). At least this way I will have to use some brain cells.

You've been warned! To emphasize the point, here is a quote from the "Ingres r3 Getting Started" document (part of the downloaded documentation package):

"Note: Make sure you thoroughly understand the issues regarding system resources, storage locations, and other configuration parameters before installing Ingres. If you are not sure, have someone more knowledgeable in these areas perform the following tasks."

I have not even checked or tested that system requirements are good enough - here is the briefest of specs:
I'm using Compaq nc6120, with 2G RAM, inside a VMWare virtual machine with Fedora Core 4.

So, to the installation.

Step 1 ================================================================
Download the following files from CA site: http://opensource.ca.com/projects/ingres/

ingres-3.0.2-105-pc-linux-i386.tgz
ingresr3documentation.tar.gz

You will need to register with CA first.

Step 2 ================================================================
Prepare for the installation: create 'infrastructure' and unpack the tarballs.

# mkdir -p ingres/documentation

Unpack the documentation. I chose to separate the documentation (PDF files) from the installation software.
# cp ./ingresr3documentation.tar.gz ingres/documentation
# (cd ingres/documentation; tar xzvf ingresr3documentation.tar.gz)

Unpack the Ingres 3 software
# cd ingres;
# tar xzvf ingres-3.0.2-105-pc-linux-i386.tgz
# cd ingres-3.0.2-105-pc-linux-i386

Step 3 ================================================================
Perform the installation

Need to run the Licence scripts first, to accept the conditions. The main database installation script won't work until you do this one first. Trust me, I know this from experience ;-)

# ./ingres-CATOSL

Can now run the main Ingres 3 installation script.

# ./*install
INSTALL OUTPUT: Invoking RPM...
INSTALL OUTPUT: Preparing... ################################ [100%]
INSTALL OUTPUT: System user ingres has been created but no password has been set.
INSTALL OUTPUT: Please set the password to enable login for this user.
INSTALL OUTPUT: 1:ca-ingres ################################ [ 6%]
INSTALL OUTPUT: II_INSTALLATION configured as II.
INSTALL OUTPUT: 2:ca-ingres-dbms ################################ [ 12%]
INSTALL OUTPUT: Setting up Ingres Intelligent DBMS, this may take some time...
INSTALL OUTPUT: See /opt/CA/IngresII/ingres/files/install.log for more info.
INSTALL OUTPUT: 3:ca-ingres-net ################################ [ 18%]
INSTALL OUTPUT: Building the password validation program 'ingvalidpw'.
INSTALL OUTPUT: Could not compile ingvalidpw: Using executable from the distribution instead.
INSTALL OUTPUT: Executable successfully installed.
INSTALL OUTPUT: 4:ca-ingres-qr_run ################################ [ 24%]
INSTALL OUTPUT: 5:ca-ingres-abf ################################ [ 29%]
INSTALL OUTPUT: 6:ca-ingres-c2audit ################################ [ 35%]
INSTALL OUTPUT: 7:ca-ingres-das ################################ [ 41%]
INSTALL OUTPUT: 8:ca-ingres-esql ################################ [ 47%]
INSTALL OUTPUT: 9:ca-ingres-ice ################################ [ 53%]
INSTALL OUTPUT: Setting up Ingres Web Deployment Option, this may take some time..
INSTALL OUTPUT: See /opt/CA/IngresII/ingres/files/install.log for more info
INSTALL OUTPUT: ngres release 3 has installed successfully.
INSTALL OUTPUT: The instance will now be started...
INSTALL OUTPUT:
INSTALL OUTPUT: Starting Ingres, instance

So, the following Ingres packages have been installed:

# rpm -qa | grep ingres
ca-ingres-net-3.0.2-105
ca-ingres-das-3.0.2-105
ca-ingres-odbc-3.0.2-105
ca-ingres-tuxedo-3.0.2-105
ca-ingres-CATOSL-3.0.2-105
ca-ingres-dbms-3.0.2-105
ca-ingres-qr_run-3.0.2-105
ca-ingres-c2audit-3.0.2-105
ca-ingres-esql-3.0.2-105
ca-ingres-jdbc-3.0.2-105
ca-ingres-ome-3.0.2-105
ca-ingres-star-3.0.2-105
ca-ingres-vision-3.0.2-105
ca-ingres-3.0.2-105
ca-ingres-abf-3.0.2-105
ca-ingres-ice-3.0.2-105
ca-ingres-rep-3.0.2-105
ca-ingres-documentation-3.0.2-105


Service 'rc' file has been installed:

# ls -l /etc/init.d/ingres*
-rwxr-xr-x 1 root root 7187 Oct 20 19:27 /etc/init.d/ingresII

Indeed: the service list in the runlevel 5 lists the following:

Ingres II name server (iigcn) - running
Ingres II recovery server (dmfrcp) - running
Ingres II DBMS server (iidbms) - 1 running
Ingres II Star server (iistar) - 1 running
Ingres II Net server (iigcc) - 1 running
Ingres II Data Access server (iigcd) - 1 running
Ingres II JDBC server (iijdbc) - not active
Ingres II RMCMD process (rmcmd) - running
Ingres II archiver process (dmfacp) - running

And, sure enough, loads of Ingres processes are running:

# ps -ef | grep ingres
ingres 25691 1 0 19:34 pts/1 00:00:00 /opt/CA/IngresII/ingres/bin/iigcn II
ingres 25805 1 0 19:34 pts/1 00:00:02 /opt/CA/IngresII/ingres/bin/iidbms recovery (dmfrcp) II
ingres 25925 1 0 19:34 pts/1 00:00:01 /opt/CA/IngresII/ingres/bin/dmfacp II
ingres 25935 1 0 19:34 pts/1 00:00:19 /opt/CA/IngresII/ingres/bin/iidbms dbms (default) II
ingres 26079 1 0 19:34 pts/1 00:00:00 /opt/CA/IngresII/ingres/bin/iigcc II gcc
ingres 26135 1 0 19:34 pts/1 00:00:00 /opt/CA/IngresII/ingres/bin/iigcd II gcd
ingres 26189 1 0 19:34 pts/1 00:00:02 /opt/CA/IngresII/ingres/bin/iistar star (default) II
ingres 26270 1 0 19:34 pts/1 00:00:02 /opt/CA/IngresII/ingres/bin/rmcmd II

That's all for this posting - next time, I'll see if my common sense approach worked. Chances are that many settings, variables, locations (possibly even additional libraries, etc) will have to be tweaked, adjusted, added...

Why am I so pessimistic? Well, it's just that it's been going too well so far! ;-)

Commenting out a block of lines in a script

Everybody knows of the '#' character as the character allowing to comment out lines that are not meant to be interpreted by the shell in a shell script.

Well, how cumbersome and untidy it ends up sometimes! Imagine you have a block of code, perhaps a big for-do-done loop, or a complex if-elif-else-fi statement and you need to, temporarily, take it out of the script for some reason. Inserting hash in front of each line, and then removing it again when done with whatever test you wanted to conduct is just plain messy.

Ok, you could argue that this can be done nicely in vi :

First, identify line numbers with:

:set nu

Then:

:10,30s/^/# /

will insert the hash at the begining of line, on lines 10 through 30, but even this is long-winded, and if you are using another editor, then the method may not be available at all.

The alternative is to use a combination of the 'do nothing' command: the colon command, and the 'here document'. Imagine the following construct:

: <<>
....
....
....
BLOCK-OF-LINES-COMMENTED-OUT

You can easily move the two lines up and down the script, and isolate the blok of line you don't want to 'participate' at run time.

How does it work? The colon command means 'do nothing', but generate exit status 0, success. The way we are using it here is: 'do nothing, and whilst doing this ;-) use stream of data provided as 'here document'. This embedded data is not not interpreted by the shell at run time, so it hides our code!

The case in date

Tha date command, one of the eldest tools in Unix has got one quirk that escaped me all those years. As usual with these things, it was a delegate who pointed this out to me, with a wry smile.

A bit of background first. If you need to extract hour and minutes from the date command, you'd probably use:

$ date +%H:%M
14:48

The little 'inverted polish logic' (is that why they keep telling me these things? ;-) ) is if you try to use the AM/PM indicator, achieved with the 'p' options.

Try the following:

$ date +%H:%M\ %p
14:49 PM

Then try:

$ date +%H:%M\ %P
14:49 pm

Don't know the background to this, but it does appear a deliberate tease ;-)

Pleasures of a tight-fit hangar

A Pup (150) embracing a Robin (400) 

Still the best kept secret?

I'm amazed how many people are still not aware of what Linux is! And I don't mean any Fred (sorry to all Freds out there!) off the street. I mean IT professionals, who come and join our UNIX classes. OK, so it was a Unix Fundamentals I'm referring to in particular; but even then, I would expect a bit more appreciation of the players in the industry.

We give away a Knoppix disk on our UNIX/Linux courses. Frequently people ask - "what's that?". In itself, this is not an unexpected question, after all Knoppix is not a distribution talked about in board rooms. But when I answer: "it's a distribution of Linux, which...." and I still get a follow up "what's that?", then I do despair!

Playing with pdksh redirection

We often test the understanding of the command line scan by asking if the following will work:

$ > /tmp/file1 2> /tmp/file2 (*)

This week, been faced with questions exploring this further. For example, notice that:

$ >log ( ps; who; df )

will work, so will

$ > log10 < /etc/hosts (while read -r line; do print $line; done )



(*) It will (as long as file exists). The shell opens all three files during the scan, and cat will be the only word remaining on the command line. In simple terms. the shell will look after the 'connectivity' between cat's I/O streams and the files.

SCO is fighing back ?

I hear that SCO are teaming up with MySQL! Must check it out - should we await MySQL license requirement soon ? ;-))

Daily Hintlet
The ps command is one of the most underused tools! Experiment with -o, which allows you to format the output. For example, try the following:

$ ps -C ksirtet -o pid=

Obviously, if you are not currently playing tetris, substitute ksirtet for another active process ;-)

Terminal capabilities

I remember going back years and years, having to use an application which was meant to be displayed on a Prestel terminal. Prestel is a simple screen protocol, of 40 columns x 24 rows, still used in some places (mostly in Europe). It's got some interesting features, such as fixed colours for certain facilities. Can't remember much of the detail anymore, but I'm sure that, for example, the background colour is like choosing an old Ford - any colour you want, as long as it is black. Quite peculiar.

This was the project, however, that tought me a lot about terminfo database and tools associated with creating and manipulating the terminal capability files. The terminfo database is in fact a collection of hundreds of individual files, each containing a description of one terminal type. These are data files, you can't 'touch' them with cat or more, but feel free (as root) to convert it into text with:

# infocmp /usr/share/terminfo/v/vt100 > /tmp/vt100.txt

If you needed to create a new terminal file, you'd pick up the most suitable (similar) existing one, modify it (using data from the hardware manual provided with your new terminal) and submit it into the terminfo system with the tic command (terminfo compiler).

These days one gets 'drivers' for every screen one can think of (and few more), but understanding of the terminfo still helps - as you can then implement scripts, menus, etc, using some of the more advanced screen control techniques.

For example, many people know of the tput command, that allows to manipulate the screen on an ad hoc bases. This command uses the capabilities file indicated by your TERM variable. To check, type:

$ echo $TERM

If you are looged into Linux, that is very likely xterm, and the file used is /usr/share/terminfo/x/xterm.

If you've never seen tput in action, log onto any Unix or Linux box, open any terminal session and type:

$ tput bold; echo hello world; tput rev; echo and again; tput rsg0

The fun starts when you begin to utilise some of the more obscure features, such a colour or placement control. Try the following:

$ red=$(tput setf 4)
$ echo "${red}$(tput cup 10 20)hello world"

The setf argument allows to control the character colour, whereas cup x y will send the cursor to column x row y of your screen.

DailyHintlet
To find out more about the screen capabiliites and arguments to tput look at the manual pages for terminfo, as well as tput. There is a lot more useful stuff there, for example there is an explanation of why setf 4 might mean red on one ocassion but green on another!

DailyHintlets

I found that doing a decent write-up on any particular thought or a 'discovery', etc takes time, and for that reason I often delay posting, to the point that the thought is gone, and opportunity for recording it missed.

So, from now on, in addition to sometimes longer ellaborations, I will now have a snappy entry, called "DailyHintlet". They will usually be one-liner solutions, examples, thoughts, etc. Often they will be bits of information I've learned from the delegates!

Despite the name, I don't really expect them to be posted daily, just liked the title ;-)

DailyHintlet

In GRUB configuration, the scope of the password directive changes, depending on where it is applied. If applied in the general definitions section, it controls ability to edit the configuration file. If applied within a stanza paragraph, it controls the access to (ability to boot) that kernel.

(Ghanny from CompuCentre - thank you for correcting my lab solutions)

grep with -E option

One of the most confusing things newbies face when learning Unix is having to grasp regular expressions. Having gone through the pains of learning the globbing (wildcards) and other shell specials, only a couple of days later we throw them all into air (characters, that is - not the delegates) and try to catch them again!

It's interesting to watch the delegates try to rationalise it all... What usually finishes them off, is the use of the counting mechanism, whereby one can specify how many instances of a character or a pattern is expected.

For example: to locate lines (in datafile) with at least 4 characters using grep could be done simply with (remember, in regular expressions dot means any one character):

$ grep '....' datafile

But if you want to locate lines containing exactly four characters, grep is taking on a bit more complex detail. The counting mechanism used needs to be 'switched on' with the use of the backslash.

$ grep '^.\{4\}$' datafile

The delegates are by this stage happy (ish) with the 'anchoring' characters (^ and $) and know that this means tying the pattern to the beginning and end of line, respectively. What freaks them out is that \{ \} notation.

I like using it, because it allows me to complete the explanation of the backslash usage in Unix - you use it to toggle a special-meaning of a character. We normally use it to turn the special meaning of a character off. In the counting notation of grep, we use it for the exactly opposite purpose. Here, it means turn the special meaning of the curly brackets on (and make it count the character specified in front of it).

But, I will have to find a different example to illustrate the use of the backslash; I just can't justify for much longer keeping quiet about the -E option one can use with grep. This allows the use of the set of extended regular expressions, in which counting is part of the default, and the { } notation is taken to mean counting without any additional measures...

Therefore, the last example could be re-written with:

$ grep -E '^.{4}' datafile

Much simpler, and less confusing...

Why sudden change of heart? Had an e-mail from a delegate along the lines of: "Alina, did you know that..."

Memory stick Duo and HP 6120...

...do not work together!

HP 6120 comes with a fancy digital card reader port. It takes every card. Apparently. What was not documented very well, is that the new format memory sticks, such as the Magic Gate ones, do not (and will not) work. Indeed, some of the sites on the web warn you against using them, as you can corrupt the card.

So, I thought I'd get a PCMCIA card, with the memory stick adapter. Guess what! I have found a way of predicatably generating a blue screen of death on XP!

And that's after being convinced by many that this is now in the past. It actually is blue, even if it doesn't last long, not even long enough for me to read the info (not that I would really want to).

So, off to Jessops again, to get a USB external reader. More bits to carry.

Lack of standards is getting at me.

I'm back

A gap, mostly down to my swapping the hardware, and spending ages on installing, re-installing and configuring machines.

I now have a brand new (QA's, I must add) HP6120, with all the spec necessary to run VMWare workstation 5, with RHEL4, Fedora 4, Solaris 10 and SuSE 9.3 running withing it.

Quite impressive! Of those listed above, Solaris 10 is having the greatest problems. Mostly because there isn't yet vmware-tools program for Solaris, which means that things like screen resolution, cut and paste and sharing directories with other virtual machines is at best limited, at worst non-existent.

But very impressive otherwise. The 2Gig RAM is a must for this kind of configuration, and with it you can have all guests open and running.

Good stuff!

Counting patterns in KSH

I was explaining the pattern matching in the shell earlier today, and used a very unfortunate example for explaining the "exactly one" pattern:

if [[ $var == @([0-9])% ]]; then...

I have (correctly) explained the above pattern as exactly one digit followed by a percent sign. One of the delegates asked me: why use the fancy characters? Wouldn't the following:

if [[ $var == [0-9]% ]]; then...

be the same? And if so, what's the point of the additional characters! Got me going for a moment! The answer is in the poorly chosen example. Although it works, one would never use the 'exact one' counting in relation to a single pattern specification.

The @(...) notation is used when you need to 'count' alternative patterns, as in:

if [[ $var == @(+|-)[0-9] ]]; then...

Without the @ character, the brackets needed for enveloping the alternative patterns would not work:

if [[ $var == (+|-)[0-9] ]]; then...
ksh: syntax error: '==' missing second argument

And without the brackets altogether:
if [[ $var == +|-[0-9] ]]; then...

gives:
ksh: syntax error: '|' unexpected operator/operand

Bootable Knoppix

I'm teaching the shell scripting course, and as usual I'm being asked if we have access to the Internet.

We normally don't allow the Internet in the classroom (unless it is needed for the course, of course). We just need to take away a temptation of browsing the net in the middle of a session, as we (the lecturers) are a vain lot, and can take such behaviour personally ;-)

Now, however, my approach has changed! This week, having asked Del to connect the class to the net, instead sending the deles to our Internet cafe's, I told them to reboot their boxes from the Knoppix CD, and use it for e-mail at breaks or lunch. The process of rebooting the machines is short enough to make it viable, but long enough to stop them from abusing it.

That means the best of both worlds! No interruption during lectures, yet they can access their e-mail. Will be recommending this method to our MS-Windows lecturers ;-))

found it...

Everything shows OK from other PCs. How stupid of me! It's the local caching on my laptop.
All that was needed was a page reload!