OviOS Linux 2.00 Admin Guide

OviOS Linux version 2.xx Antares

      OviOS is a linux based storage OS, with out-of-the-box support for iSCSI, SMB and NFS.

      The ovios-shell is the storage management shell for OviOS Linux. 
      The ovios-shell is designed to assist in setting up the
      server, and requires very little Linux/UNIX/storage knowledge. 
      Most commands are interractive and if an error occurs, the messages can be found using 'ovilogs' 
      Do NOT create pools, volumes, LUNs and targets using the Linux bash shell. 
      These will not work well in ovios-shell. 
      Objects created and managed with the ovios-shell have specific configurations and specific options set.

      There are 4 steps to create a storage server using the ovios-shell.
      1. Configure the network.

      1.1. Use 'bondadm' to create aggregated links if desired.
      The syntax is: # bondadm -n name -i interface1 -i interface2 -m mode 
      Use 'linuxcmd' to drop to the Linux terminal and type 'bondadm'
      In ovios shell, bondadm opens it's own CLI menu. It requires only
      the arguments. for EX:
      ovios-shell> bondadm
      To exit the bondadm CLI enter quit.
      bondadm > -l
      1 ==>> eth0 : STATE: up : MODE: balance-rr 0
              Physical interfaces:
              1 ==> eno16777736
              2 ==> eno67109432

      To exit the bondadm CLI enter quit.
      bondadm >
      Example creating a bonded interface: 
   > bondadm -i eno1345 -i eno1245 -i eno5620 -n eth0 -m 0
   Where: eno1345, eno1245 and eno5620 are physical interfaces.
    eth0 is the name given to the new aggregated link 
    0 is the mode chosen. 
      The ipmitool command from ovios-shell works the same way. 
      1.2. Use 'netsetup' to setup the network, by setting a static IP or enabling DHCP.

      2. Create the RAID sets or storage pools.

      There are multiple settings available:
                1. Add Raid0 (Stripped devices.) pool
                2. Add Raid1 (Mirrored devices. mirror) pool
                3. Add Raid5 (raidz) pool
                4. Add Raid6 (raidz2) pool
                5. Add Raid10 (mirrored stripped) pool

      2.1. Choose the number corresponding to the desired RAID level.
      Enter a name for the pool, and enter the devices to create the pool. 
      Use 'storage' to get a list of all available devices.
      The system accepts devices which can be found in /dev/, such as sda, sdc, sdd etc. 
      Or use the full path, such as /dev/disk/by-path/disk1 etc
      For simplicity, the admin should use only dev names (sdb sdc etc). 
      When the pools are imported during a boot up,
      the zfs-admin script will import the devices by path.

      2.2. Add spares and log devices. 
      To add spare disks, just type "spare disk1 disk2 disk3..", at the end of the disk list, 
      where disk1,2,3 are the dev names.
      For EX: "Enter devices to add to the pool: sdb sdc sdd sde spare sdf sdg sdh"
      In a mirrored pool (Raid10) you can add spares at the end of the last mirror created, 
      or at the end of each mirror.
      You can add LOG devices as well. 
      EX: "Enter devices to add to the pool: sdb sdc sdd log sde spare sdg sdf"
      Example for a RAID10 setup:
      Enter devices to add to the pool: sdb sdc log sdd spare sde
      Enter devices to create the mirror: sdf sdg spare sdh log sdi

      3. Set up SAN storage

      Use OviOS Linux as an iSCSI Server. OviOS LUNs can be used in UNIX/Linux environments, Windows or VMWare. 
      To setup an iSCSI server, the admin must create iSCSI targets, LUNs and map the LUNs to targets.
      Start the iSCSI server with 'iscsi start'

      3.1. Create iSCSI targets.
      Run 'target create' and enter a name for the target when prompted to do so. 
      Enter only a custom name, the IQN and server identifier will be added automatically. 

      3.2. Create LUNs. Run 'lun create' and create a LUN. A LUN must be created in a storage pool. 
      Enter the storage pool name when prompted.
      A LUN can be thin or thick provisioned. Thin provisioning is disabled by default in OviOS because the target doesn't support the DISCARD SCSI command yet. 
      However, thin LUNs can be created.
      I strongly recommend to create only thick LUNs to have a better grip on the space available.
      If targets already exist, run 'lun_setup' to create and map the LUN in a single command.
      Map the LUN to a target using 'lun map'. A unique LUN ID will be assigned automatically. 
      This LUN ID will not be changed.
      After LUNs have been mapped, run 'iscsi reload' to make them available.
      A LUN can only be mapped to one target. 
          Use meaningfull target names. 
          Use only thick LUNs. 
          Use target ACL (when creating targets the command will ask for initiator IP 
          or IQN) to control the iSCSI sessions

      4. Set up NAS storage.

      OviOS can be used to share volumes via NFS and SMB.
      Create a volume in a pool with 'vol create' to start.

      4.1. NFS server
      Start with 'nfs start' Supports all NFS versions, up to 4.2.
      Run 'nfs_export' to export a volume via NFS. If client IPs must be enterred, use colon to separate them.
      EX: etc
      'nfs ss' shows which option a volume is exported with. 'vol list' also shows this.

      4.2. OviOS SMB server
      Start with 'smb start'

      4.2.1. With local authentication. 
      Run 'smbuseradd' to add an SMB user. The command will ask for the path where the user's home will be.
      This should be a volume and the path MUST be in this format: /pool/volume 
      Do not leave out the first slash and don't append a final slash. 

      4.2.2. With remote authentication.
      Join the OviOS SMB Server to a DC. Run 'smbjoindc'. This is a TUI tool which requires: 
          - A DC admin user
          - a path to users' homes in this format: /pool/volume
          - DC domain name
          - AD name 
          - DNS, ntp and DC IP
      'smb ad-users' will list all available AD users.
      4.2.3. Share volumes via SMB. Run 'smb_export'. This tool will require a user 
      (can be local smb user or AD user) which will be granted admin rights to this share. 
      This user will be used to manage the share permissions.       
      For more advanced permissions, the admin can use the linux terminal and 'setfacl' to set user permissions. 
      note: ovios is not an SMB user, therefore this user won't be accepted. Only users listed with 
      'smb ad-users' and 'smbuserlist' can manage shares.
      The share name will be in this format by default: pool_volume
      The share name can be changed manually. Edit the file in /var/lib/samba/usershares/pool_volume. 
      Change the file name to the desired share name as well!

    5. Setup replication

    Replication can be setup between 2 OviOS nodes or one OviOS node and another 
    Linux system which runs zfs on linux. If the destination is not an OviOS system,
    it must have the 'zfs' and 'zpool' binaries in /usr/sbin.

    Before setting up replication, make sure that: 

            The ssh passwordless authentication
            must be set up for user root between the nodes.
            Make sure the source node can ssh into
            the destination node with user root without password.

            When running a full replication with retadm initiate,
            the source pool and dataset MUST exist.
            On the destination, only the pool must exists,
            the destination dataset will be created by retadm.
            If the destination dataset exists, the command will
            print an error and exit.

    The replication tool is called 'retadm'.

            retadm initiate - initiates a new full replication

            retadm inc <poolname/dataset> - runs an incremental 
                        replication for poolname/dataset

            retadm modify <poolname/dataset> - allows to modify 
                        the destination hostname or IP

            retadm reset <poolname/dataset> - resets all replication 
                        properties for poolname/dataset

            retadm status - displays a current status for all datasets 
                        in the system, on which replication is enabled. 
                        The ones without replication enabled are ignored

            retadm help - prints the help manual.

    5.1. Set up initial replication:
        Run 'retadm initiate' . The interactive tool will ask for the destination hostname or IP,
        will check if the destination is reachable and authentication works.
    ovios-shell> retadm initiate
    Initiates a new replication for a vol or LUN.
    Requires ssh authetication to already be configured.
    Enter the destination host or IP: filer3
    Checking filer3 is available
    filer3 found and is accessible  
    Checking if passwordless authentication works to filer3
    Passwordless authentication tested successfully to filer3
    Enter the source poolname: pool01

    Enter the volume name or LUN name: lun01

        When entering the poolname which will be used on destination, 
        retadm will check if this pool exists. It MUST exist.

        The datastore on the destination MUST NOT exist. Just give it a name 
        and it will be created. 

    5.2. Schedule incremental replication:

    Once the initial replication completes, the admin can setup a cron job to
    schedule replication for multiple volumes and LUNs.
    A messages will be logged in /var/log/ret when a replication started and 
    when it finished.

    Or use 'retadm status' to determine the status:

            Replication status for: retpool/lun001
                Destination: filer2 : retpool/lun001
                Status: completed_2016-08-31_08-39-56
                Basesnap: retpool/lun001@ovios_repl_inc-2016-08-31_08:39:53

    The format is to schedule an incremental replication:

    */15 * * * * /sbin/retadm inc datapool/share01

    This will run an incremental replication every 15 minutes for datapool/share01

    5.3. Break a replication:

    Run 'retadm reset <poolname/dataset>' to clear all replication settings 
    for this dataset.
    It will reset the settings to default on the source and destination, and also
    delete the base snapshot on the source and destination. 
    The volume / LUN will not be deleted on either source on destination.
    6. Disaster Recovery

      The replicated shares and LUNs are always available on the destination system.
      The destination system's configuration can be synced with the source automatically, 
      so that in a DR scenario all shares, LUNs and iSCSI Targets are available immediately.
      For this to work, the pool name on the source and destination must be identical, 
      and all shares and LUNs on the destination must have the same name as the source.
      If these requirements are met, run 'sync-config dest <destination host or IP>
      This will sync all users, groups, hosts, SMB settings, iSCSI settings to the 
      The network settings are , for obvious reasons, not synced, as the source has a different
      hostname and IP addresses. 
      If the destination host requires different local settings, then the admin must prepare the
      config manually. Create local users, create SMB users, join to an AD and so on. 
      The script /etc/sysconfig/ovios/ovios_restore can be used on the destination
      to automatically create the targets and map the replicated LUNs to the targets, by not removing
      any targets which exist already on the system. However, the LUNs will be mapped with 
      their original LUN IDs, so in this case the admin must be carefull not to have duplicate LUN IDs,
      and correct those manually if needed.

      Advanced Settings not available from ovios-shell. These require some advanced Linux knowledge.
      Feel free to send me an email at ovios.storage@gmail.com or ovi@ovios.org if you need help with these settings.