Wednesday, February 17, 2010

Free up Cache memory in Linux

Kernels 2.6.16 and newer provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory.

To use /proc/sys/vm/drop_caches, just echo a number to it.

To free pagecache:

# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches


This is a non-destructive operation and will only free things that are completely unused. Dirty objects will continue to be in use until written out to disk and are not freeable. If you run "sync" first to flush them out to disk, these drop operations will tend to free more memory.
Posted by at No comments:
Labels:

Tuesday, February 2, 2010

Veritas VxVm cmds (4.0) for Solaris Boot Disk Management

VxVM Manual ch 4, 5 
The correct procedure for remirroring bootdisk from bootmirror so that
they are nearly identical in paritioning scheme should be:
(after many tries):

/etc/vx/bin/vxdisksetup -i c1t0d0 format=sliced
       Add disk c1t0d0 to vertias control, using sliced format for boot-able di
sk.

vxdg -g rootdg adddisk bootdisk=c1t0d0

vxmirror -g rootdg bootmirror bootdisk
       This mirror partition and cyl in right seq.

/etc/vx/bin/vxbootsetup -g rootdg rootdisk
       just in case vtoc or boot sector info is missing.

----

Some more notes from the try, vxassist command used as per book suggestion.

fix later, from try 2 on oaprod2

 vxplex -g rootdg dis rootvol-01
 vxplex -g rootdg dis var-01 u01-01 swapvol-01
 vxedit -g rootdg -fr rm rootvol-01
 vxedit -g rootdg -fr rm swapvol-01 var-01 u01-01
 vxdg -g rootdg rmdisk bootdisk
 vxdisk rm c1t0d0

/etc/vx/bin/vxdisksetup -i c1t0d0 format=sliced
 Add disk c1t0d0 to vertias control, using sliced format for boot-able disk.

vxdg -g rootdg adddisk bootdisk=c1t0d0

vxassist -g rootdg mirror swapvol bootdisk &
 Mirror swap slide
 try mirror swap first so that it will start at cyl 2
 It won't put any partition info in vtoc yet.

/etc/vx/bin/vxrootmir bootdisk
 mirror the root partition, it will need slide 0 be avail.
 This would probably work, except somehow var was placed in disk earlier
 than root, so will need to do that first. 
 vxassist -g rootdg mirror var  bootdisk &
 vxassist -g rootdg mirror u01  bootdisk &

 So instead, use vxmirror...
vxmirror -g rootdg bootmirror bootdisk
  This mirror partition and cyl in right seq.
 oaprod2 seems to be fine.



vxmirror   : mirror root disk

Missing!!  Try in test "4"
/etc/vx/bin/vxbootsetup -g rootdg rootdisk
 Configure bootable info on vxvm boot disk in rootdg disk group.
 This include boot sector and solaris VTOC info that match
 location of root, swap, /usr, /var.
 Optional media name(s) at end (rootdisk) specify action to be carried out
 on specified disk only (if omitted, done for all disk in dg).

 More info:
 http://www.sun.com/blueprints/0800/vxvmref.pdf

 vxbootsetup would be needed for vxassist, and slide indeed show up in vtoc/format.
 but vxassist method req more work to copy partition in the right cyl
 manner.  Maybe if bootmirror was created using vxassist, then that
 would have been good.  But right now, vxmirror works great
 and don't really need vxbootsetup (at least in 4.0)

---

VxVM commands on oaprod1, clean tba

 544  vxmirror -g rootdg c1t1d0 c1t0d0
 545  vxmirror c1t1d0s2 c1t0d0s2
 546  vxdg -g rootdg c1t1d0s2 c1t0d0s2
 547  vxmirror -g rootdg c1t1d0s2 c1t0d0s2
 548  vxmirror -g rootdg c1t1d0s2 c1t0d0s2
 549  vxmirror -g rootdg c1t1d0s0 c1t0d0s0
 550  vxprint -hrt
 556  vxdisk list
 564  vxdmpadm -h
 565  vxtask list
 566  vxprint -hrt
 567  vxdisk  list
 576  vxdisk list
 577  man vxdisk
 578 

vxdisk list
 display all disk managed by Veritas
vxdisk -s list
 much more detailed version than above
vxprint
 print disk slice/mirroring info, plex,vol, etc
vxprint -hrt
 diff version than above, useful to see boot disk mirroring location
 on disk.


vxvol -g oracledg stopall
 stop all veritas volume on the specified disk group oracledg.
 all volume must be unmounted first

vxdg deport oracledg
 export the disk group oracledg
 this would make all the disk/volume of the said disk group
 to be available for import by other hosts.
 optional -h  parameter tell who is expected to do import
 
vxdg list
 list all disk group
vxdisk  -o alldgs list
 scan all disk, incl those not currently managed by veritas.
 those with disk group name inside () are exported dg, ready for import.

vxdg import oracledg
 import the said disk group oracledg

vxvol -g oracledg startall
 start all disk in said disk group oracledg
 This will make all vol in said disk group to be avail for mounting
 and general use.
 Veritas may have some locking mechanism based in scsi3 to fence off
 multiple host from importing a disk group, even if forceful.
 (as maybe problem on SAN disks shared in cluster).
 Check this first though.

----

vxddladm addjbod VID=DGC pagecode=0x83 offset=8 length=16
 disable Veritas DMP (dynamic MultiPath) and use EMC PowerPath driver instead
 (EMCpower pkg need to be installed).

vxddladm listjbod
 Display current settings

vxddladm rmjbod vid=DGC
 Undo use of EMC PowerPath, and let Veritas DMP do its thing again.

Veritas DMP will continue to be used, PowerPath sits at lower layer and intercept the calls
to the different disks.  Veritas will know multiple path to the LUN, and it will know they are
the same.  If fiber is removed, DMP won't know as power path work behind the scene.  Syslog
will log error.  After using vxddladm addjbod, reboot machine w/ reconfigure to ensure all devices are seen.


vxdmpadm getdmpnode enclosure=Disk
 list paths to various disks/luns


May  3 15:05:15 oaprod1 vxio: WARNING: VxVM vxio V-5-0-181 Illegal vminor encountered

The error is said due to vm disk starting up before vxconfigd, but my experience is that vxconfigd is already running.
Other things to check:
There is no files named:
/VXVM-UPGRADE/.start_runed
/etc/vx/reconfig.d/state.d/install-db


ls -l /dev/vx/dsk/
reminor disk group whose import is in conflict w/ existing disk group.

ls -l /dev/vx/dsk/oracledg
brw-------   1 root     root     259,52001 May  4 17:10 u02
brw-------   1 root     root     259,52000 May  4 17:10 u03
                                    ^^^^^
                                    ^^^^^   52000 is the minor number, shown also in vxprint -l oracledg

vxdg -g oracledg -f reminor 53011 # change the minor number of a disk group,
     # must be done while disk group is mounted
     # but changes take effect only in the next import run
     # and this minor number belongs to the disk group, which get carried to
     # another node when it is imported there.





Other commands

vxassist

/usr/sbin/modinfo | grep vx : look for loaded veritas modules (to see if veritas was running)

vxlicense -p  : list licenses available in the system
vxlicense -c  : interactive program to enter veritas license key
   : license files are stored in /etc/vx/elm/
/usr/sbin/vxdisk list : list veritas disks, status

/usr/sbin/vxprint -ht  : show some long config,
/usr/sbin/vxprint -m :
   : veritas setup, etc



vxdiskadm  : encaptulate root disk
vxmirror   : mirror root disk
vxassistmirror 

vxrecover : rebuild? failed vx slide
vsedvtoc : edit the veritas "fdisk/format" info

vxvol stop : stop all veritas volumes on root disk
vxedit -r : remove volume


Veritas Cluster Server

Volume Management consideration under VCS

Disabling io fencing:

/etc/rc2.d/S97vxfen stop
echo "vxfen_mode=disabled" > /etc/vxfenmode
/etc/rc2.d/S97vxfen start
Above will still load driver, which allows Veritas Cluster VM to work w/o io fencing disks,
but driver need to be loaded.  This is supported for certain CVM config, but not for RAC.

/var/adm/messages that kernel driver is loaded.
AND error on console that it is disabled (where command was started).

Best way is still to turn iofencing off completely in the init script.

modinfo | egrep vx\|gab\|llt
31  12490f9  26abc 258   1  vxdmp (VxVM 4.1z: DMP Driver)
32 78216000 20f869 259   1  vxio (VxVM 4.1z I/O driver)
34  126d9f0   1499 260   1  vxspec (VxVM 4.1z control/status driver)
277 7859f205    c7b 261   1  vxportal (VxFS 4.1_REV-4.1B18_sol_GA_s10b)
278 78a24000 170105   8   1  vxfs (VxFS 4.1_REV-4.1B18_sol_GA_s10b)
282 78bc2000  22467 266   1  llt (LLT 4.1)
283 78be6000  46a88 267   1  gab (GAB device 4.1)
284 78c2e000  39491 268   1  vxfen (VRTS Fence 4.1)



IOFencing.
driver get loaded by kernel.
it search for vxfendg to find which disk group to use, then
run vx... cmd to generate /etc/vxfentab, which has list of device path for LUN beloging to io fencing dg.
4.0 use /dev/rdsk/cXtXdXsX, 4.1 use the multipath devices, such as /dev/rdsk/emcpowerXc
the driver is much more robust than the vxfentsthdw script.
4.0 vxfentsthdw -g vxfencoorddg fails, though driver should be all good.
4.1 vxfentsthdw -g will work correctly using emcpowerX devices and know that they may not be the
same device path on different nodes.  4.1 resolved all io fencing issues found in 4.0.

4.1 vxdisk -o alldgs list will also show the emcpowerX as device name, instead of generic DISK_X.
The naming now is at the mercy of PowerPath, veritas see them just as solaris format command see them.
It may still not be persistent binding, but at least easy corss ref b/w veritas, solaris format, and info
presented in EMC Navisphere.  No ASL (Array Support Lib) was needed.

VCS Cluster Config

gabconfig -a : display link config info.  a = ??gab port, ie loaded ok by kernel.  b = iofencing port.  h = cluster port.

GAB Port Memberships
===============================================================
Port a gen   1ea001 membership 01                             
Port b gen   1ea00f membership 01                             
Port h gen   1ea012 membership 01                             


vxfenadm -i  /dev/rdsk/emcpower0c
  Display serial number of the device (LUN, disk)
vxfenadm -g  /dev/rdsk/emcpower0c
  Show IO Fencing info


graceful shutdown of cluster

hastop -all # stop vcs for the whole cluster, ready for both machine to shutdown.
hastop -local # stop vcs on local machine only, it will stop the services, no migration by default.
hastop -local -evacuate # stop vcs, migrate (evacuate) service to another node
   # evacuate a single node, just single node clean exit out of cluster.

hastatus # monitor cluster status, no arg act like tail -f
  -sum # display summary and exit.

lltstat  # general summary
 -nvv # see cluster interconnect link info (heartbeat).


hares -online Mount_u02 -sys oaprod1  # online the give resource at the specified system
      # resource name is as per config (Main.cf)
      # resource and group name are listed by hastatus cmd.
hares -offline Oracle_oaprod -sys oaprod2 # offline the whole resrouce group on the specified system
      # migration to another node will NOT happen for -offline.
hagrp -switch oracle_group -to oaprod1  # switch a service group to the specified system
hares -modify Oracle_oaprod Owner oracle # change resouce=Oracle_oaprod attribute=Owner new_value=oracle
haconf -dump      # save vcs config to Main.cf (edited via special command)
      # do not edit Main.cf while cluster is up, it will be ignored.
haconf -dump -makero    # equiv of "close config" of hagui, config still kept by conf editor


seq of offline commands:
hares -offline Netlsnr_oaprod -sys oaprod2
hares -offline Oracle_oaprod  -sys oaprod2
hares -offline Mount_u02      -sys oaprod2
hares -offline Mount_u03      -sys oaprod2
hares -offline Volume_u02     -sys oaprod2
hares -offline Volume_u03     -sys oaprod2
hares -offline DiskGroup_oracledg -sys oaprod2 # some sort of high level container wrapper.
hares -offline IPMultiNICB_oaprod -sys oaprod2

hares -clear   Oracle_oaprod  -sys oaprod1
hares -online  Oracle_oaprod  -sys oaprod1 # bring up oracle service group, w/ all deps
hagrp -switch  oracle_group   -to  oaprod2


---
config eg, for adding oracle test group.

haconf -makerw
hagrp -freeze oracle_group
hares -modify Oracle_oaprod User veritas_monitor
hares -modify Oracle_oaprod Pword veritas_password
hares -modify Oracle_oaprod Table monitor
hares -modify Oracle_oaprod MonScript "./bin/Oracle/SqlTest.pl"
hares -modify Oracle_oaprod DetailMonitor 1
haconf -dump -makero
hagrp -unfreeze oracle_group


---


config commands (typically located in /opt/VRTS/bin):

hacf -verify /etc/VRTSvcs/conf/config/
 verify that the main.cf config file is correct, parseable.

haconf -makerw
 turn config to be read write, so that changes can be made via haclus
haconf -dump -makero
 save and close config from rw, must remember to do this, or else reboot will have issues!
haclus ...
 change cluster config param.  (CLI change instead of gui).

hauser -add vcsuser
 Add a new user that can use hagui, it will prompt for the new password of the new user.

hauser -modify Administrators -add vcsuser
 The new user is placed in admin group so that full control is granted.
 Best way to add admin when password is forgotten :)

hagui GUI, java, for monitor and making changes to cluster

VCS config files

/etc/VRTSvcs/conf/config/main.cf
 config file for vcs, usually changed using hagui or haclus command.
 Once cluster is live, config is in memory, and this file is only backup.
 Changes to it will be ignored if cluster is up. 
 Cluster start does read this file, so easy manual chage of config if cluster is down.


/etc/init.d/vx*
vxvm-relocover
 starts several deamon, which also take argument and email root at local machines. 
 change these!


3 files in /etc need to be copied to each of the node in the cluster
(rsh of install should create these if doing multinode install w/ install script).


/etc/llttab ::
set-node oaprod1   # diff for each node, reflect local node name
set-cluster 1
link ce1 /dev/ce:1 - ether - -
link ce3 /dev/ce:3 - ether - -
link-lowpri ce0 /dev/ce:0 - ether - -

/etc/llthosts ::
0 oaprod1
1 oaprod2

/etc/gabtab ::
/sbin/gabconfig -c -n2
Posted by at No comments:

How to find/display your MAC Address: Unix/Linux/Windows/Mac

Linux
  • As the root user (or user with appropriate permissions)
  • Type "ifconfig -a"
  • From the displayed information, find eth0 (this is the default first Ethernet adapter)
  • Locate the number next to the HWaddr. This is your MAC address
The MAC Address will be displayed in the form of 00:08:C7:1B:8C:02.
Example "ifconfig -a" output:
eth0 Link encap:Ethernet HWaddr 00:08:C7:1B:8C:02
inet addr:192.168.111.20 Bcast:192.168.111.255 Mask:255.255.255.0

...additional output removed...

Solaris/SunOS
  • As the root user (or user with appropriate permissions)
  • Type "/sbin/ifconfig -a"
  • From the displayed information, find the Ethernet adapter (it will probably be called le0 or ie0)
  • Locate the number next to ether. This is your MAC address
The MAC Address will be displayed in the form of 0:3:ba:26:1:b0 -- leading zeros are removed. For this example, the actual MAC Address would be 00:03:ba:26:01:b0.
Example "ifconfig -a" output:
le0: flags=863 mtu 1500
inet 192.168.111.30 netmask ffffff00 broadcast 192.168.111.255
ether 0:3:ba:26:1:b0

FreeBSD/NetBSD
  • As the root user (or user with appropriate permissions)
  • Type "ifconfig -a"
  • From the displayed information, find the Ethernet adapter (the name changes based on the Ethernet card installed)
  • Locate the number next to the HWaddr. This is your MAC address
The MAC Address will be displayed in the form of 00:08:C7:1B:8C:02.
(Using the command"dmesg"will also display the MAC address -- along with a lot of other information)
Example "ifconfig -a" output:
ed0: flags=8843 mtu 1500
inet 192.168.111.40 netmask 0xffffff00 broadcast 192.168.111.255
ether 00:08:C7:1B:8C:02

...additional output removed...

OpenBSD
  • As the root user (or user with appropriate permissions)
  • Type "netstat -in"
  • From the displayed information, find the Ethernet adapter (the name changes based on the Ethernet card installed)
  • Locate the number below Address. This is your MAC address
The MAC Address will be displayed in the form of 00:08:c7:1b:8c:02.
Example "netstat -in" output:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls
fxp0 1500 00:08:c7:1b:8c:02 4112773 0 224501 0 0

...additional output removed...

Caldera/SCO UnixWare/OpenUNIX
  • As the root user (or user with appropriate permissions)
  • Type "ndstat"
  • From the displayed information, find net0 (this is the default first Ethernet adapter)
  • Locate the number below MAC Address in use. This is your MAC address
The MAC Address will be displayed in the form of 00:00:c0:88:0a:2e.
Example "ndstat" output:
Device MAC address in use Factory MAC Address
------ ------------------ -------------------
/dev/net0 00:00:c0:88:0a:2e 00:00:c0:88:0a:2e

...additional output removed...

HP-UX (HP UNIX)
  • As the root user (or user with appropriate permissions)
  • Type "/usr/sbin/lanscan"
  • From the displayed information, find lan0 (this is the default first Ethernet adapter)
  • Locate the number below Address. This is your MAC address
The MAC Address will be displayed in the form of 0x000E7F0D81D6 -- the leading hexadecimal indicator should be removed. For this example, the actual MAC Address would be 00:0E:7F:0D:81:D6.
Example "lanscan" output:
Hardware Station Dev Hardware Net-Interface NM Encapsulation Mjr
Path Address lu State NameUnit State ID Methods Num
2.0.2 0x000E7F0D81D6 0 UP lan0 UP 4 ETHER 52

...additional output removed...

IRIX (SGI UNIX)
  • IRIX 4.01 or later
    • As the root user (or user with appropriate permissions)
    • Type "netstat -ia"
    • From the displayed information, find the Ethernet adapter (the name changes based on the Ethernet card installed)
    • Locate the number below Address. This is your MAC address
  • Alternate Method
    • Typing "/etc/nvram eaddr" should also show the MAC address
The MAC Address will be displayed in the form of 00:00:6b:71:1a:6a.
Example "netstat -ia" output:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
ec0 1500 nowhere warum 6514913 10234 184317 0 13513
192.168.111.90
00:00:6b:71:1a:6a
...additional output removed...

NeXTStep
  • As the root user (or user with appropriate permissions)
  • Type "/sbin/ifconfig -a"
  • From the displayed information, find the Ethernet adapter (it will probably be called le0 or ie0)
  • Locate the number next to ether. This is your MAC address
The MAC Address will be displayed in the form of 0:0:f:a1:75:a0 -- leading zeros are removed. For this example, the actual MAC Address would be 00:00:0f:a1:75:a0.
Example "ifconfig -a" output:
le0: flags=863 mtu 1500
inet 192.168.111.70 netmask ffffff00 broadcast 192.168.111.255
ether 0:0:f:a1:75:a0

AIX (IBM UNIX)
  • As the root user (or user with appropriate permissions)
  • Type "netstat -ia"
  • From the displayed information, find the Ethernet adapter (the name changes based on the Ethernet card installed)
  • Locate the number below Address. This is your MAC address
The MAC Address will be displayed in the form of 00:09:6B:51:1f:79.
Example "netstat -ia" output:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
ec0 1500 nowhere flotsam 5514233 11434 101317 0 14113
192.168.111.95
00:09:6B:51:1f:79
...additional output removed...

Tru64 UNIX (Digital UNIX)
  • As the root user (or user with appropriate permissions)
  • Type "netstat -ia"
  • From the displayed information, find the Ethernet adapter (the name changes based on the Ethernet card installed)
  • Locate the number below Address. This is your MAC address
The MAC Address will be displayed in the form of >00:00:F8:1a:73:da.
Example "netstat -ia" output:
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
ec0 1500 nowhere jetsam 5514233 11434 101317 0 14113
192.168.111.95
00:00:F8:1a:73:da
Windows NT/2000/2003/XP
  • Open the command prompt
  • From the command prompt type "ipconfig /all"
  • (If the information scrolls off the screen you can type "ipconfig /all|more")
  • Find the network adapter you want to know the MAC address of
  • Locate the number next to Physical Address. This is your MAC address
The MAC Address will be displayed in the form of 00-02-2D-11-55-4D.
Example "ipconfig /all" output:
Ethernet adapter Wired:
Connection-specific DNS Suffix . : roundfile.com
Description . . . . . . . . . . . : ORiNOCO PC Card (5 Volt)
Physical Address. . . . . . . . . : 00-02-2D-11-55-4D

Windows 95/98/ME
  • Click the Start Menu, then select Run
  • Type in "winipcfg" and click OK
  • Select the appropriate Ethernet adapter
  • Locate the number next to Adapter Address. This is your MAC address
The MAC Address will be displayed in the form of 00-10-5A-44-12-B5.
Example "winipcfg" output:
WINIPCFG Display

DOS/Windows 3.11/Windows for Workgroups
  • From the Command Prompt
  • Type in "msd"
  • Select the Network... button
  • Locate the number next to Net01 ID:. This is your MAC address
The MAC Address will be displayed in the form of 0004E25AA118.
Example "msd.exe" output:
MSD Display

Macintosh OS X
  • Wired (MacOS 10.2 and earlier)
    • From the dock, select System Preferences
    • Select Network, Select Location, Select Interface
    • Select TCP/IP Tab
    • Locate the number next to the Ethernet Address. This is your MAC address
  • Wired (MacOS 10.3 and later)
    • From the dock, select System Preferences
    • Select Network, Select Location, Select Interface
    • Select Ethernet Tab
    • Locate the number next to the Ethernet ID. This is your MAC address
  • Wireless (AirPort)
    • From the dock, select System Preferences
    • Select Network, Select Location, Select Interface
    • Select AirPort Tab
    • Locate the number next to the AirPort ID. This is your MAC address
The MAC Address will be displayed in the form of 00:0D:93:13:51:1A.

Macintosh OS w/ Open Transport (Pre-MacOS X)
  • From the Apple Menu, select Control Panels
  • Open either the Appletalk or the TCP/IP Control Panel
  • From the Edit Menu, select User Mode
  • Change the mode to Advanced or Admin
  • Click the Info Button (lower left hand corner)
  • An AppleTalk or TCP/IP window will pop up
  • Locate the number next to the Hardware Address. This is your MAC address
The MAC Address will be displayed in the form of 00 0D 93 13 51 1A.

Macintosh OS w/ MacTCP (Pre-MacOS X)
  • Make sure CAPS Lock is not on
  • Make sure the Macintosh is connected to an Ethernet network
  • From the Apple Menu, select Control Panels
  • Open the MacTCP Control Panel
  • Hold down the Option Key and click the Ethernet icon
  • Locate the number next to the Hardware Ethercard Address that appears beneath the icon. This is your MAC address
The MAC Address will be displayed in the form of 00 0D 93 13 51 1A.
Posted by at No comments:
Labels:
Subscribe to: Posts (Atom)