Installation & How-TO

GEMs Master Install

    Installing the required packages
    Install the Master
    Setup the Database
    Setup Cron Jobs
    Setup Rsync Mirror (optional)

GEMs Endpoint Install

    Setup USE flags for installing PHP
    Install and configure Endpoint

GEMs Master and Endpoint Upgrade

    Master Server upgrade
    Endpoint Upgrade

GEMs Monitoring Agent Install (Optional)

    Install and configure the Agent

Quick Start tutorial for GEMS

---

This assumes you have a working webserver with PHP and MySQL or PgSQL
You will need PHP-Pear and following Pear Packages:
    emerge PEAR-PEAR
    emerge eix
    pear install Pear
    pear install Cache_Lite
    pear install XML_Tree
    pear install XML_RSS
    pear install XML_Parser
    pear install MDB2
    pear install MDB2_Driver_mysql  <OR> pear install MDB2_Driver_pgsql

    Optional for the monitoring console:
    emerge rrdtool
    mkdir /path/to/gems/monitoring
    chown webserver_user:webserver_group /path/to/gems/monitoring
    chmod ug+wr /path/to/gems/monitoring

Edit your /etc/php/*/php.ini so allow_url_fopen = On

1. Unpack the tar file to your desired web directory
1. tar -C /path/to/webdir  -xzvf gems_master-v1.x.tar.gz
2. Edit the config.php file
1. $log_path --> Where the GEMS application will write it's logs
2. $log_file --> Logfile name
3. $logging --> 1 turn on logging; 0 turn off logging
4. $portage_dir --> Usually /usr/portage
5. $cache_dir --> Where to write cache files, should be a temp diretory
6. $cache_life --> Seconds before expiring the cache
7. $port --> The port number to use when connecting to endpoints (endpoint config must match)
8. $servername --> For the address the endpoints will report to, rather than use what PHP finds
9. $db_type --> Now there is support for both mysql and pgsql
10. $host --> For the database
11. $dbuser and $dbpassword --> login info for MySQL or PostGres
12. $db --> The name of the database for GEMS (should be gems by default)
3. Create the database
1. MySQL
1. Adding a database user to MySql:(optional if you have already setup users)
   mysql -uroot -p
   CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'newpassword';
   grant select, insert, update, delete, create, drop, file, index, alter, create temporary tables, create view, show view on *.* to 'testuser'@'localhost' identified by 'newpassword';
   
2. mysql -uroot -ppassword < gems.sql
2. Postgres SQL
1. If you need to create a database user(optional if you already setup users):
1. From a regular shell
   createuser -A -d -P -E -U postgres -W <user>
   Enter password for new user:
   Enter it again:
   Password:
   CREATE USER
   This will create a user in postgres that cannot create databases or add db users
2. Again from a regular shell
   createdb -U <user> -W gems
   This creates a database called gems, as user <user>
3. Now enter the postgres shell
    psql -U <user> -W gems
   Password:
   Welcome to psql 8.0.8, the PostgreSQL interactive terminal.
   
   Type:  \copyright for distribution terms
          \h for help with SQL commands
          \? for help with psql commands
          \g or terminate with semicolon to execute query
          \q to quit
   
   gems=> \i gems.pgsql
   gems=> \q
   This starts a postgres shell in the gems database as user <user>
   \i gems.pgsql loads the gems postgres DB schema
   \q exits the postgres shell
4. The user <user>, the password, and the database (gems in this case) are what you will use in gems master config.php file.
2. This postgres info was taken from the Gentoo Postgres Guide. Check it out for further info.
4. Setup the distfiles sym link  (for the dist files mirror).
1. cd /path/to/webdir/gems 
2. ln -s /usr/portage/distfiles  distfiles
3. This is the path for GENTOO_MIRRORS= in /etc/make.conf on the endpoints
1. GENTOO_MIRRORS="http://example.com/gems/" (you don't need to add distfiles to the path, its assumed)
5. Add these entries to your root crontab 
1. 30 3 * * * /usr/sbin/eix-sync
2. 15 0 * * * cd /path/to/gems && /usr/bin/php /path/to/gems/glsa-update.php
3. * * * * * cd /path/to/gems && /usr/bin/php /path/to/gems/wait_queue_watch.php 2>&1>/dev/null
6. Setup your RSYNC mirror
1. To make this server an rsync mirror as well, follow section 2 of this guide: http://www.gentoo.org/doc/en/rsync.xml
7. The webserver must be added to the portage group (for fetching distfiles)
1. gpasswd -a <apache | lighttpd> portage
8. Login to GEMS http://gems.example.com/gems/
1. Default login --> user: admin  password:admin
2. Users Menu --> Show users --> Edit admin --> Change the default password
3. Add a user
9. You have to setup some endpoints before you can continue, so you're done with the server for now. 

GEMS Endpoint setup

This assumes PHP is installed with CURL
USE flags for PHP should be posix, pnctl, cli, sockets, curl for gemsd to function

1. Unpack the tar file to a temp directory
1. tar -C /tmp  -xzvf gems_endpoint-v1.x.tar.gz
2. Run inst_ep.sh 
1. ./ep_inst.sh or /bin/bash ep_inst.sh
2. Edit /etc/make.conf and set GENTOO_MIRRORS="http://example.com/gems" whatever you decided on in step 4.3 above
3. Go back to the GEMS web page and select Endpoint Utils > Register/Unregister
4. Enter the endpoint's IP address or DNS name (what you can ping it by)
1. Click Register
2. The endpoint will now be registered to THIS GEMS server.  Other GEMS server will not be allowed to connect to it.  

GEMS Master Server and Endpoint upgrade

Master Server

1. Backup your exisit gems directory structure
1. cp -a /path/to/gems /path/to/gems.bak
2. Update the database schema for MySQL
1. mysql -uroot -p gems <mysql_db_update.sql
3. Configure config.php.  Don't reuse the old one, new features have been added  
4. Add a new cron entry (does not have to run as root)
1. * * * * * /usr/bin/php /path/to/gems/wait_queue_watch.php

Endpoint

1. Run the ep_install.sh script
2. You can opt not to copy the new config file in, your old one will work fine

GEMS Monitoring Agent Setup (optional)

1. Unpack the tar file to a temp directory
1. tar -C /tmp -xzvf gmond_agent-v1.x.tar.gz
2. Run install.sh
1. ./install.sh or /bin/bash install.sh
2. The installer will add a user to your system called gmond, with a uid 9803.  You can change these if you want, the installer asks you (you don't need to unless you have a good reason)
   The user will belong to the users group, and will have a home dir of /dev/null, and login shell of /bin/false
3. You will be asked for the url to your GEMs site. This is so the agent knows where to report in.
4. If the url was correct, and the installer can verify it (using wget) it will continue 
5. The last step complies and installs WWW-Curl, the Perl Curl binding. 
3. You can now start the daemon, and add it to the startup
1. /etc/init.d/gmond start
2. rc-update add gmond default
4. If your GEMs monitoring site is setup correctly, you should see that the agent reported in.  It takes a few mintues to see data in the graphs
5. To view the console, login to your gems server, select the Alerts/Monitoring Menu, and click Monitoring Console.
   

GEMS Usage

Once you have your server setup, and your endpoints, you can start using gems.  Here is an explanation of the pages.

1. Portage Menu
1. Search --> Searches Portage on this server
2. Browse --> Browse Portage on this server
3. Distfiles --> Shows all distfiles on this server
4. Searches and browsing portage will give discriptions of the ebuild, along with some links:
1. Browse Portage --> Brings you back to the main portage page
2. Fetch... --> Brings you to a page that allows you to fetch the package to this server's mirror
3. Deploy --> Brings you to the deploy page to install the package on endpoints
2. Endpoint Utils Menu
1. Register/Unregister --> For adding or removing endpoints from this GEMS server
2. Run Command --> Run a command (as root) on an endpoint.  When the command is done, you'll see the return code, and the last line of the command's output
3. Send File -->  Allows the Endpoint to pull files from a repository on your GEMS server.  This page also has a feature to populate the repository through http upload
4. Endpoint List -->  Show all of the endpoints registered, and some information about them
5. Inventory Collection --> Enter the names of endpoints to collect Portage software inventory 
6. Inventory Search --> Search by computer name or ebuild name
3. Alerts/Monitoring Menu
1. GEMS Alerts --> GEMS alerts, like install failures, inventory errors, and install success
2. GLSA Alerts --> Shows the current GLSA notices from Gentoo.org
3. Monitor Deployments --> Shows current GEMS processes, like installs and inventory collection
4. Monitoring Console --> If you use the Monitoring Agent, you can view system stats for your servers here.  This feature is new, and currently shows CPU use, System load, and Memory use
5. View GEMS log --> Log of GEMS application activity 
4. Users
1. Add users --> Add a user to the GEMS application
2. Show users --> View and edit current GEMS users

How to Install software on an endpoint

This is a quick tutorial on how to get started with software installs to endpoints using bash as an example

1. Login to GEMS and from the Portage menu, select Search (you can also use browse if you want)
2. Search for bash
3. Look for the bash shell (app-shells/bash) in the results
4. Click Fetch Verbose This will fetch the package to this server, along with all dependencies
5. When the fetch completes, click Deploy down at the bottom
6. Enter your endpoint names in the targets box (Endpoints must be registered)
7. (Optional) If you are installing a package that has a daemon you can use the next few boxes to add it to the run levels:
1. Add daemon to init --> Yes or No, this is like doing rc-update add daemon default
2. Run Level --> Default or Boot
3. Daemon name --> The name of the daemon, like when using rc-update
8. Click deploy package, you will see a connection to each endpoint.
9. Go into Alerts/Monitoring and click on Monitor Deployments.  You will see the endpoints currently installing displayed in this list.  Refresh it frequently and you will see the list get shorter as endpoints finish installing.
10. Go into Alerts/Monitoring and click on GEMS Alerts.  You will see the status of you deployments.