UNIX notes

Main Site : CNK's space : Unix : Unix Notes

Supervise Notes

Supervise is a replacement for inittab written and distributed by D. J. Bernstein. Source code and some likely looking man pages can be found at http://cr.yp.to/daemontools.html but I have never been able to get things to work based on those instructions. Here are the things that I gleaned from conversations with sysadmins who can actually make the thing work:

Setting it up

Do the following as root:
  1. Make a directory for your service in /var/service/ - initially, name this directory .yourservice - the dot at the front keeps svcscan from seeing it (in case svscan is already running on your box).
  2. In that directory you will need an executable script named "run". run is a shell script that will start up whatever service you want supervise to start, stop, and keep running. In our case, this is generally something like this:
         #!/bin/sh
    
         exec /home/aol33/bin/nsd-oracle -i -u nsadmin -c /home/aol33/makewaves-dev.ini -g arsdigit
      
  3. You can run supervise one of two ways - run a separate supervise process for each of the servers (start them individually from the command line using 'supervise /var/service/yourservice') or you can use svscan to supervise everything in a particular path of subdirectories. Let's assume that you want to use svscan. Move your . directory where it can be seen - `mv .yourservice yourservice'. If svscan is already running, then that is all you need to do. If you need to start service scan use the following:
         nohup svscan /var/service/yourservice & > /dev/null 2>&1
      
    The nohup part of the command is so that hangup signals are ignored - the command can continue running in the background after you log out. The first & puts this process in the background. The rest of the gibberish is about redirecting output (CNK look this up). Note: once supervise is running it will make a subdirectory in you directory that it will use to keep track of things it needs to know. `ls supervise` will usually show files: control, lock, ok, and status

Stopping it

  1. Move the yourservice directory to .yourservice (so svscan does not see it any longer)
  2. svc -dx /var/service/.dir - will make svscan forget all about your service.

How to set up rhosts

Do this so that you don't have to type your password to get stuff from a remote computer. Cribbed from Ron's post http://www.arsdigita.com/bboard/q-and-a-fetch-msg?msg_id=0004gA

Greg felt that ssh with RSA authentication is the best compromise between security and funtionality for now. To get CVS working correctly from a running nsd process takes a little more work. I've integrated this into the vc module and re-written file-manager to use vc for all of its CVS transactions. This is being folded into ACS 3.3, but you'll probably need to ugprade to get it all working.

For the record, if you need to authenticate other hosts, here are the steps:

  1. on the remost host login as nsadmin and run ssh-keygen with no passphrase to create ~/.ssh/identity and ~/.ssh/identity.pub
  2. copy identity.pub to the repository host and append it to ~nsadmin/.ssh/authorized_keys
  3. chmod 400 ~nsadmin/.ssh/authorized_keys

The user "nsadmin" will now be able to connect using ssh from the remote host to the repository host without having to supply a password.

You can do this for your own account to stop from going insane while working with remote repositories.

CNK 7/29/02 Figure out how to use ssh-agent/ssh-add to add ssh keys to shells.

Sending email with telnet

Telnet to some mail server (your choice) with port 25. Then:
helo
mail from: someone@some.address
rcpt to: whoever@you.want
data 
Put your message here. 
It can cover multiple lines. 
To end and send, put at . on a line all by itself. 
.

Sending email from the command line

/usr/lib/sendmail -bm -t -f from@example.com < text_file_with_mail_message
or
mail [-s subject] [-c cc-addr] [-b bcc-addr] to-addr... 

Network magic

To find out what ports have bound listeners on your box, on linux do:
     sudo netstat -ltp --numeric-ports
On OsX, netstat takes different options and there doesn't seem to be an equivalent for the -p flag (show me what process this is). So instead, use lsof:
     sudo lsof -i -P

One way to test out name-based virtual hosting configs is to edit your /etc/hosts file so that your machine sends trafic to your site before you switch the DNS listing. I have had trouble getting this to work on my Mac. Apparently the problem is not using the correct command to flush my DNS cache:

      dscacheutil -flushcache

The usual way to find out the IP address of some other host is to use nslookup or dig. However, those to commands go directly to DNS and do not pay any attention to the machine's resolver configuration. So they are useless for telling you what IP you will actually be connecting to. So to test if changes you make in your /etc/hosts file (for example) are working, you need to use ping:

$ ping xyz.example.com
PING xyz.example.com (172.16.17.204) 56(84) bytes of data.
64 bytes from v-acs-db1-prod.acs.caltech.edu (172.16.17.204): icmp_seq=1 ttl=64 time=0.597 ms

Ruthanne's wireless security notes

Flushing DNS cache

To get your Mac to pick up changes you make to /etc/hosts, you need to flush the DNS cache. How to do this has changed several times. The incantation for Yosemite is:

    sudo discoveryutil udnsflushcaches

Backups

Look at M Rubel's backup scripts. Also look at rdiff: rdiff-backup.stanford.org.

Mutt Notes

Bash + RedHat 8.0 tip

Under bash 2.05, which is shipped with RHL 8.0, file completion on a symlink that points to a directory doesn't add a slash to the end by default. Normally, if you type
  cd /sof
and there's a directory /software, and nothing else that starts with "/sof", it'll complete to
  cd /software/
at which point you could continue typing, e.g
  cd /software/stow/blahblah
In bash 2.05, this only happens if /software is actually a directory; if it's a symlink that points to a directory, it instead completes to
  cd /software
so you have to explicitly type the trailing slash -- but only for symlinks, not actual directories.

As you might guess, this is insanely annoying, at least if you use symlinks that point to directories. But good news: It turns out that this is a readline variable, which you can customize by adding

  set mark-symlinked-directories On
to your .inputrc. Yay! Thanks go to Josh Smith for this info.

Fedora Core Terminal Annoyances

I kind of like the tabbed terminal - but not at the expense of not being able to use M-b and M-f in my emacs buffers. There is an option under the Edit menu to edit keyboard shortcuts - but it doesn't seem to let you change these two options. However, it will let you "Disable all menu access keys". This alleviates the annoying menu popping when in the Gnome terminal program so I can use emacs in peace. (Besides, I have screen if I need > 1 screen within my single window.)

Resetting terminal

Don't you hate it when you cat (or grep) a binary file and your shell goes all wierd? Instead of killing it, just type:
   tput reset

Handy Unix Commands

lsof
ls of open files - shows you what file handles are open and who is hanging on to them
netstat -p
Shows what process is bound to a particular port (need to be root to see it) I usually use 'netstat -ltp --numeric-ports' to see what is listening via tcp on my box
fuser
Gives you the id of the process that is using a resource. For example 'fuser 111/tcp' will show you the process id for the service bound to port 111 via tcp
pidof
Find the pid of _____ name some program ______
ps -ef --forest
My favorite set of arguments to ps - you get the tree view but also can see parent id to look for the infamous "parented to 1" Oracle client processes. The other handy one is ps axwf
pgrep and pkill
From the man page
       pgrep  looks  through  the currently running processes and   
       lists the process IDs which matches the selection criteria      
       to stdout.  All the criteria have to match.  For example,   

       pgrep -u root sshd

       will  only  list  the  processes  called sshd AND owned by         
       root.  On the other hand,

       pgrep -u root,daemon

       will list the processes owned by root OR daemon.

       pkill will send the specified signal (by default  SIGTERM)
       to each process instead of listing them on stdout.
find
There are a lot of teriffic options to find - but the one thing I find annoying is all the "permission denied" error messages when doing a traverse as an unprivileged user. To get rid of those use
  find . -name foobar -print 2>/dev/null
grep
grep is great - but sometimes it would be nice to have some context around the matches, like you can ask for in a diff. Answer: pass a number argument to grep, e.g., -3 gives you 3 lines of context before and after the match. OR -A 2 -B 1 to get 1 line before and 3 lines after the match. (And, of course, one usually wants case insensitive matches -i and line numbers -n)
Checking file integrity
$ md5sum filename
60ca4fa286cb45aafbfd01e0f052c5c3 filename

Then compare it with the published MD5 hash for the file. Let's you
download the typical 3 mondo .isos of a Linux distribution, to burn to 3
CDs. Copy the three md5sums into a file, with the filenames, into a file,
and call it checksums.txt:

76ef22495d186580e47efd8d7a65fe6b  yarrow-i386-disc1.iso
fd23fe32fafe7557f5d1fa1d31100580  yarrow-i386-disc2.iso
6a26b34069639d0c31465d4079a8e1b2  yarrow-i386-disc3.iso

Put this in the same directory as the .isos, then run

$ md5sum -cv checksums.txt
yarrow-i386-disc1.iso OK
yarrow-i386-disc2.iso OK
yarrow-i386-disc3.iso OK

Generating your own md5sum is ever so easy:

$ md5sum filename
44d305fdb74dd643c8d4f760024e9fee  filename
id
What is my guid, uid, and what groups do I belong to?
$ id
uid=1000(carla) gid=1000(carla) groups=1000(carla),20(dialout),24(cdrom),
25(floppy),29(audio),30(dip),44(video),105(windows),432(usb),
1001(cdrecording),1008(share)

Handy SysAdmin Commands

/sbin/lsmod
List of loaded modules, use count, and referring modules. On my linux box this is identical to 'more /proc/modules'
lspci
Gives information about all PCI interfaces (USB devices, IDE and ISA devices.
/sbin/fdisk -l
List all information about your hard drives - physical characteristics as well as partition table info.
dmesg | more
Use this to read the boot messages - after booting.

How to make logrotate rotate at midnight

Logrotate is nice - and if you install apache using rpms on a RedHat system, it is configured to rotate access logs daily. Unfortinately, not at the start of the day. To fix that, you need to edit /etc/crontab so the 'daily' tasks happen at midnight. The default file is:

$ more /etc/crontab 

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly
Change the second line in run-parts to
00 * * * * root run-parts /etc/cron.daily

Shell scripting

How do you randomize when a command runs from cron? So that something runs around 2:30 - but not all precisely at 2:30 PM. There are a couple of options:
      14 30 * * * sleep `perl -e 'print int(rand 10)'` ; /path/to/mycommand.sh
Or within the bash script that you are going to run:
      #!/bin/bash
      # Waits a maximum of 32767 / 50 seconds (~10 min)
      seconds=`expr $RANDOM / 50`
      sleep $seconds
N.B. the $RANDOM variable only exists for bash - not for vanilla Bourne shell

System monitoring

Using top

Found out that you can customize top's output - interatively or with a config file (either /etc/toprc or individually ~/.toprc). Made a version of top that is just as I like it and saved the configuration with "W". The other things I need to remember are that you can change the sort order of top on the fly:

    P - sort by CPU usage (default)
    M - sort by memory usage
    N - sort by pid (numerically).
    A - sort by age (newest first).
    T - sort by time / cumulative time.
c toggles full command information on and off. And S toggles whether the times given include time taken by all forked children (even those that have finished).

Memory usage with free

Useful info on how to interpret the output of free from http://lists.slug.org.au/archives/slug/2002/07/msg00513.html:

On Wed, Jul 24, 2002 at 11:57:34AM +1000, Melinda Taylor wrote:
> 
> I just noticed after using dump that 100% of the memory in my system is
> now in use. I check this also on my linux laptop, after using
> 'dump' my 412 MB ram had only 12k free.
> 
> The command free shows:
> 
>              total       used       free     shared    buffers
>              cached
> Mem:        514328     502352      11976          0       4476
> 345684
> -/+ buffers/cache:     152192     362136
> Swap:       875500          0     875500

This is actually fine.

The line that matters in free is the "-/+ buffers/cache" line, which
shows how much memory is available to applications (by adjusting for how
much memory is used for the disk cache).  So looking at it again, we
see:

> -/+ buffers/cache:     152192     362136

So your apps are using about 150Mb, and you still have about 360Mb free.

If you look at the top line, you'll notice about 340Mb is used for
"cached", which make sense after running dump, because you've done lots
of disk activity.  Linux keeps as much of the disk in memory as
possible, because RAM is much faster than disk.  However, if more RAM is
needed for a program, Linux will discard parts of its cache to do so.

Another sign that you don't need to worry is the "Swap" line shows 0
bytes used, which means your system hasn't swapped anything out to disk,
which is only done when real physical RAM is running low.

I hope this makes it a little clearer.  There's a lack of good, clear,
non-technical documentation about what those numbers actually mean.

Regards,

-Andrew Bennetts 
 Wed Jul 24 12:13:02 2002

Misc.

How do you scroll up on the terminal?

How do you convert a man page to html? for example, the httperf man page:

groff -t -e -mandoc -Thtml /software/man/man1/httperf.1 | col -bx > httperf.html

Other Sections


cnk@ugcs.caltech.edu