• Category Archives Gentoo
  • Gentoo Linux distribution

  • Unix » Linux » Gentoo
  • Updating Gentoo

    Today, I started the task of updating my home_network gentoo hosts since its been a while.

    What a mess I have run into, baselayout2+openrc has been marked stable, luckily I caught the newsitem not to reboot before fixing the /etc/init.d directory with etc-update or similar tools.

    Next up, I have tons of broken packages and revdep-rebuilt wont get me out of the mess at once. So, I have to manually track packages who’s dependency libs are broken (DirectFB I am looking into you!) , which thankfully is not all that difficult because of the q-toolkit (qdepends and qdepends -r). Still, i would like to use 1-command to update my whole host as advertised on www.gentoo.org.

     


  • Finally, the time has come… CFEngine

    The amount of work that has not been automated bores me to death.

    Imagine changing your own user password every month (!) on x-hundredth hosts manually for different reasons.

    Of course, you can script it with an expect script and ssh-keys, but its still not ideal. What if your account is locked on one of them or the password changed manually at one time for a reason. then the expect script fails and the host in question is not completed.

    First up, to determine policies and to add Gentoo’s portage so Cfengine recognises it in package management. Nothing too elaborate, just a simple setup that works.

    package_changes               => "individual";
    package_list_command          => "/usr/bin/python -c 'import os; os.system(\"/bin/ls -d /var/db/pkg/*/* | cut -c 13-\")'";
    package_list_name_regex       => ".*/([^\s]+)-\d.*";
    package_list_version_regex    => ".*/[^\s]+-(\d.*)";
    package_installed_regex       => ".*";                  # all reported are installed
    package_name_convention       => "$(name)";
    #package_list_update_command    => "/bin/echo --sync";           
    package_list_update_command    => "/usr/bin/eix-sync";           
    package_list_update_ifelapsed     => "14400";                 
    
    
    package_add_command        => "/usr/bin/emerge -q --quiet-build";
    #package_add_command         => "/bin/echo Installing";
    package_delete_command      => "/usr/bin/emerge --depclean";
    package_update_command      =>  "/usr/bin/emerge --update";
    package_verify_command      => "/usr/bin/emerge -s";
    package_noverify_regex      => ".*(Not Installed|Applications found : 0).*";

    This causes portage to synchronise every 10 days and we use eix to speedup certain things.

    Next up, we will save this file but not yet activate it. First I used the default update_script that came with cfengine as a failsafe mechanism. Editted to suit my needs, I don’t need a

    /home/mark(etc)

    with access to localhost ;-). Although these are mark’s unix pages of course. still, as a humourous sidenote maybe. Nope, editted out this one. Next up, I changed the place where cfengine keeps its masterfiles, for security-through-obscurity reasons. Although not real security, if it throws off even 10% of scriptkiddies it means it was successful.

    When the client’s intranet is ready, I will move the cfengine off the public interface despite the fact the traffic never sees the full unprotected internet. Hackers can be even within one’s own ISP after all ūüôā

    Finally, I will configure cfengine to run shell-scripts on every host when present in a directory on its master and move the shell-script to an archive-directory along with the output logfiles.

    My basic setup is now finished and onward to a more difficult setup.

     


  • SSH Ignore (un)known host keys

    Sometimes, you wish that SSH would just connect without asking questions. For instance if you are on a trusted net where you do not need to worry about man-in-the-middle attacks.

    You can realise that wish with the following settings.

    Onetime setting as commandline argument:

    $ ssh -o UserKnownHostsFile=/dev/null \
     -o StrictHostKeyChecking=no user@192.168.0.100
    Warning: Permanently added '192.168.0.100' (RSA) to the list of known hosts.
    user@192.168.0.100's password:

    Permanent in ssh users config ~/.ssh/config

    StrictHostKeyChecking=no
    UserKnownHostsFile=/dev/null

    Explanation:

    UserKnownHostsFile sets the location of known hosts. In this case, we use /dev/null to make sure there aren’t any existing hosts with offending host keys already present.

    StrictHostKeyChecking=no means that ssh will automaticly add the key to the database (/dev/null) without asking for user confirmation.

    These 2 steps together mean that ssh will totally ignore any known or unknown host key and just login without silly questions.


  • Serviio DLNA player and my AQUOS TV

    How to configure your Serviio software to recognise the Sharp AQUOS TV.

    in the serviio software install directory, there is a directory named “config” with 2 files in it. log4j.xml and profiles.xml

    step 1) Open the profiles.xml
    Step 2) paste the “code” block from this post

    <Profile id="14" name="Sharp AQUOS" extendsProfileId="1">
       <!-- Rev. 00.20110506 (initial) - added detection, transcoding mp3 -->
       <!-- by KFL, tested on LC40LE814E fw1.08 -->
       <!-- DLNA certification http://certification.dlna.org/certs/REG52162737.pdf -->
       <!-- Supported DLNA profiles:
             Image: JPEG_MED, JPEG_SM
             Video: MPEG_PS_PAL, MPEG_TS_SD_EU, MPEG_TS_SD_EU_ISO, MPEG_TS_SD_EU_T
             Audio: LPCM
       -->
          <Detection>
          <!-- detection: User-Agent: DLNADOC/1.50 SHARP-AQUOS-DMP/1.1W -->
             <HttpHeaders>
                <User-Agent>.*SHARP-AQUOS-DMP.*</User-Agent>
             </HttpHeaders>
          </Detection>
          <MediaFormatProfiles>
             <!-- rename native mpeg2ts with h264/ac3 ac3 formats to those supported by the tv -->
             <MediaFormatProfile mime-type="video/vnd.dlna.mpeg-tts" name="AVC_TS_HD_24_AC3,AVC_TS_HD_50_AC3,AVC_TS_HD_60_AC3,AVC_TS_HD_EU">AVC_TS_MP_HD_AC3</MediaFormatProfile>
             <MediaFormatProfile mime-type="video/vnd.dlna.mpeg-tts" name="AVC_TS_HD_24_AC3,AVC_TS_HD_50_AC3,AVC_TS_HD_60_AC3,AVC_TS_HD_EU">AVC_TS_MP_SD_AC3</MediaFormatProfile>
             <MediaFormatProfile mime-type="video/vnd.dlna.mpeg-tts" name="AVC_TS_HD_24_AC3_T,AVC_TS_HD_50_AC3_T,AVC_TS_HD_60_AC3_T,AVC_TS_HD_EU_T">AVC_TS_MP_HD_AC3_T</MediaFormatProfile>
             <MediaFormatProfile mime-type="video/vnd.dlna.mpeg-tts" name="AVC_TS_HD_24_AC3_T,AVC_TS_HD_50_AC3_T,AVC_TS_HD_60_AC3_T,AVC_TS_HD_EU_T">AVC_TS_MP_SD_AC3_T</MediaFormatProfile>
             <!-- rename transcoded mpeg2ts with h264 to one of the TV's supported profiles -->
             <MediaFormatProfile mime-type="video/mpeg" name="AVC_TS_HD_24_AC3_ISO,AVC_TS_HD_50_AC3_ISO,AVC_TS_HD_60_AC3_ISO,AVC_TS_HD_EU_ISO">AVC_TS_MP_HD_AC3_ISO</MediaFormatProfile>
             <MediaFormatProfile mime-type="video/mpeg" name="AVC_TS_HD_24_AC3_ISO,AVC_TS_HD_50_AC3_ISO,AVC_TS_HD_60_AC3_ISO,AVC_TS_HD_EU_ISO">AVC_TS_MP_SD_AC3_ISO</MediaFormatProfile>
          </MediaFormatProfiles>
          <Transcoding>
             <!-- For dvr-ms files force mpeg2video transcoding to fix monotone timestamps problems -->
             <Video targetContainer="mpegts" targetACodec="ac3" forceVTranscoding="true">
                <Matches container="asf" vCodec="mpeg2video" />
             </Video>
             <!-- Transcode all h264 video with HIGH/MAIN > Level 4.1 on MPEG-TS stream with MPEG2VIDEO and ac3 audio transcoding -->
             <!--  expected profile MPEG_TS_SD_EU/MPEG_TS_SD_NA/MPEG_TS_SD_KO -->
             <Video targetContainer="mpegts" targetVCodec="mpeg2video" targetACodec="ac3" aBitrate="384">
                <Matches container="avi" vCodec="h264" profile="high" levelGreaterThan="4.1" />
                <Matches container="avi" vCodec="h264" profile="main" levelGreaterThan="4.1" />
                <Matches container="matroska" vCodec="h264" profile="high" levelGreaterThan="4.1" />
                <Matches container="matroska" vCodec="h264" profile="main" levelGreaterThan="4.1" />
                <Matches container="mp4" vCodec="h264" profile="high" levelGreaterThan="4.1" />
                <Matches container="mp4" vCodec="h264" profile="main" levelGreaterThan="4.1" />
             </Video>
             <!-- Remux all other h264 video on MPEG-TS stream with ac3 audio transcoding -->
             <!--  expected profile AVC_TS_MP_HD_AC3_ISO -->
             <Video targetContainer="mpegts" targetACodec="ac3" aBitrate="384">
                <Matches container="avi" vCodec="h264" />
                <Matches container="mp4" vCodec="h264" />
                <Matches container="matroska" vCodec="h264" />
                <!-- if audio different to ac3, must be transcoded -->
                <Matches container="mpegts" aCodec="aac" />
                <Matches container="mpegts" aCodec="mp3" />
                <Matches container="mpegts" aCodec="dca" />
             </Video>
             <!-- All Others video/audio codec will be transcoded into mpeg2ts, mpeg2video and ac3 audio -->
             <!--  expected profile MPEG_TS_SD_EU/MPEG_TS_SD_NA/MPEG_TS_SD_KO -->
             <Video targetContainer="mpegts" targetVCodec="mpeg2video" targetACodec="ac3" aBitrate="384">
                <Matches container="asf" />
                <Matches container="avi" />
                <Matches container="mp4" />
                <Matches container="matroska" />
                <Matches container="flv" />
             </Video>
             <Audio targetContainer="lpcm">
             <!-- DLNA audio: LPCM -->
                <Matches container="asf" />
                <Matches container="mp4" />
                <Matches container="flac" />
                <Matches container="ogg" />
                <Matches container="mp3" />
             </Audio>
          </Transcoding>
          <AutomaticImageRotation>true</AutomaticImageRotation>
       </Profile>
    
    

    Step 3) Restart Serviio

    If you did it correctly, you should see this line in the serviio logs:

    2011-07-12 16:56:10,535 INFO [ProfilesDefinitionParser] Added profile 'Sharp Aquos' (id=14)

    Step 5) Turn off TV, wait 5 minutes
    Step 6) delete the TV as a registered device using the serviio GUI.
    Step 7) Turn on tv

    If all went correctly, you should see the TV register itself with the Aquos profile.

     2011-07-12 17:05:47,850 INFO [RendererManager] Stored a new renderer: uuid='721aac88-42f6-435a-b61f-XXXX', name = 'Sharp Aquos', ipAddress='192.168.X.XX', profile = '14'

    Enjoy !

    Mark.


  • Lynis reports expired certificate

    Today,

    My security scanner Lynis informed me that it found an expiring certificate for my cyrus imap server. Oh noes!!!!

    So I quickly located the following instructions online:

    openssl req -new -nodes -out req.pem -keyout key.pem
    openssl rsa -in key.pem -out new.key.pem
    openssl x509 -in req.pem -out ca-cert -req \
    -signkey new.key.pem -days 999

    cp new.key.pem /etc/ssl/cyrus/server.pem
    rm new.key.pem

    cat ca-cert >>/etc/ssl/cyrus/server.pem

    This seemed unsufficient as the expiring report is : /etc/ssl/cyrus/server.crt.

    The last step however was easy, copy server.pem to server.crt and we are back in business.


  • Moving a virtualbox to real hardware part II

    After last weekend, the raw hard disk image of the virtual host has been successfully extracted. Now we will use this image, to try and find the PV’s on the disk so that we can extract those and import them back.

    1) Look for volumes on the virtual disk

    [root@hardware /vbox/HDD]# fdisk -l -u virtual.bak.vdi
    last_lba(): I don't know how to handle files with mode 8180
    You must set cylinders.
    You can do this from the extra functions menu.
    
    Disk virtual.bak.vdi: 0 MB, 0 bytes
    255 heads, 63 sectors/track, 0 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    
             Device Boot      Start         End      Blocks   Id  System
    ./virtual.bak.vdi1              63      224909      112423+  83  Linux
    ./virtual.bak.vdi2          224910     4433939     2104515   82  Linux swap / Solaris
    ./virtual.bak.vdi3         4433940    41929649    18747855   8e  Linux LVM
    Partition 3 has different physical/logical endings:
         phys=(1023, 254, 63) logical=(2609, 254, 63)

    2) Now we can extract the volume using DD, note that the first partition is not an LVM volume and the second partition is the swap file. So we need the 3d partition.

    dd if=virtual.bak.vdi of=virtual.pv skip=4433939 count= 37495710 
    
    Note: Numbers in skip and count are starting location of lvm and lenght of lvm.
    This information is gathered from fdisk.
    The start of the partition is the skip argument, the Blocks*2 is the count argument.
    
    Note2: Make SURE the bs (block size) is the default fdisk output  ( eg. Units are in 512-byte sectors )
    The sector-size determines the multiplication factor of the blocks. If the Unit size =1024 bytes then
    you do not need to multiply by 2.

    4) check for loopback devices on the system

    [root@hardware /vbox/HDD]# losetup -f
    /dev/loop0
    
    Note: If LVM probably has a lot of LVs in it you will quickly run
    out of the 8 loop devices linux allows by default. If using GRUB, you
    must pass a max_loop=X argument to set the number of loop devices
    available on boot. LILO probably has something similar.
    
    Note2: losetup -d [ to unmount the device ]

    5) Map loop device to Image

    losetup /dev/loop0  virtual.pv
    6) Scan for new physical volumes
    [root@hardware /vbox/HDD]# pvscan
      PV /dev/loop0   VG virtual_rootvg   lvm2 [17.88 GB / 2.88 GB free]
      Total: 1 [17.88 GB] / in use: 1 [17.88 GB] / in no VG: 0 [0   ]

    If you get an error like this one below:

    Note: WARNING: Duplicate VG name VolGroup00: gKOBnM-SdmK-V3SO-fI1M-Twlt-tmr6-36IoBs (created here) takes precedence over FXHljU-CSVg-zSRX-snZ2-KHxA-S5UD-3YxJYg

    That means that you have 2 pv’s belonging to a different volume group, but with the same name.

    The solution is easy, just rename the virtual_volumegroup by identifying it with its VG-ID.

    6) export VG

    
    
    [root@host system_disk]# vgexport virtual_rootvg
      Volume group "virtual_rootvg" successfully exported
    
    

    7) Import VG

    
    
    [root@host system_disk]# vgimport virtual_rootvg
      Volume group "virtual_rootvg" successfully imported
    
    

    8 ) Make volume active

    
    
    [root@hardware /vbox/HDD]# vgchange -ay virtual_rootvg
      1 logical volume(s) in volume group "virtual_rootvg" now active
    
    

    9) Try mounting the volume

    [root@hardware /vbox/HDD]# mount /dev/virtual_rootvg/lv_root /mnt/virtdisk

     

    [root@hardware /vbox/HDD]# ls -l /mnt/virtdisk/
    total 204
    drwxr-xr-x  2 root root  4096 Sep 25 04:02 bin
    drwxr-xr-x  2 root root  4096 Sep 10 07:00 boot
    drwxr-xr-x  4 root root  4096 Sep 10 07:00 dev
    drwxr-xr-x 96 root root 12288 Dec  2 04:02 etc
    drwxr-xr-x 79 root root  4096 Oct 28 00:00 home
    drwxr-xr-x 12 root root  4096 Sep 11 04:06 lib
    drwxr-xr-x  8 root root  4096 Sep 11 04:06 lib64
    drwx------  2 root root 16384 Sep 10 06:58 lost+found
    drwxr-xr-x  2 root root  4096 Sep 10 14:37 lsst
    drwxr-xr-x  2 root root  4096 Oct  6 10:40 media
    dr-xr-xr-x  2 root root  4096 Sep 10 13:05 misc
    drwxr-xr-x  2 root root  4096 Oct 10  2006 mnt
    dr-xr-xr-x  2 root root  4096 Sep 10 13:05 net
    drwxr-xr-x  2 root root  4096 Oct 10  2006 opt
    drwxr-xr-x  2 root root  4096 Sep 10 07:00 proc
    drwxr-x--- 16 root root  4096 Dec  2 14:59 root
    drwxr-xr-x  2 root root 12288 Sep 11 04:07 sbin
    drwxr-xr-x  2 root root  4096 Sep 10 14:02 scr
    drwxr-xr-x  2 root root  4096 Sep 10 07:00 selinux
    drwxr-xr-x  3 root root  4096 Sep 10 14:38 share
    drwxr-xr-x  2 root root  4096 Oct 10  2006 srv
    drwxr-xr-x  2 root root  4096 Sep 10 07:00 sys
    drwxrwxrwt 10 root root  4096 Dec  2 15:41 tmp
    drwxr-xr-x 15 root root  4096 Sep 10 07:03 usr
    drwxr-xr-x 24 root root  4096 Sep 10 07:08 var
    
    

    Now, I can unmount the volume, etc. etc. etc.:

    [root@hardware /vbox/HDD]# umount /mnt/virtdisk
    [root@hardware /vbox/HDD]# vgchange -a n
    [root@hardware /vbox/HDD]# losetup -d /dev/loop0

    Finally, I am sure that the PV is healthy and I can move it to the backup host.

    Next step, boot from Live-CD.


  • Moving Virtualbox to real hardware

    Today, my client gave me the task of moving a virtualbox machine to real hardware.

    VBox specs: 2 Gigabyte ram, 1-CPU, 80 Gig harddisk.
    Real Hardware: 8 Gigabyte ram, 4-CPU and 1 Terabyte raid5 harddisk.

    The reasons to move this host to real hardware are performance issues. The hosting machine is an old linux install which needs to be scrapped because it can’t be updated anymore. This is however impossible as long as it hosts the VBox. To work efficiently, the VBOX guest will simply become the host and not run any vbox’s anymore. The VBox operating system will be Gentoo linux.

    To aid myself in the upcoming task (and minimize weekend work time), I created an “At” job. The job will shutdown the VM and make a full harddisk clone offline and then reboot ¬†the VBox. Purpose of this exercise will be to convert the resulting VDI image to Physical Volumes of Linux’s excellent LVM management system.

    Steps:
    1) Shutdown the VBox
    2) Clone the VDI
    3) Start the VBOX
    4) Move the VDI to a Raw format using VBoxManage tool
    5) Extract the Physical volumes from the virtual disk, using dd.
    6) Copy the pv’s to a different host

    Now all the preparation is done and we can start the real work, so in the weekend:
    7) Shutdown the VBOX
    8) Shutdown the Host
    9) Boot the machine from a rescue CD with networking
    10) Wipe the Harddisk
    11) Create partitions of which is at least 1 LVM of sufficient size (100+ gig)
    12) Copy the PV’s into the new LVM
    13) Create a boot-partition and reinstall the bootsector
    14) Reboot and keep the fingers crossed weekend will begin and work end.