oda for oscar
DESCRIPTION
ODA for OSCAR. Neil Gorsuch NCSA [email protected]. Database Hierarchy. A server has one or more named databases. ODA uses the database named “oscar” by default. Each database has one or more named tables: The oscar database has “packages”, “nodes”, … - PowerPoint PPT PresentationTRANSCRIPT
National Computational Science
Database Hierarchy
• A server has one or more named databases.• ODA uses the database named “oscar” by default.• Each database has one or more named tables:
– The oscar database has “packages”, “nodes”, …
• Each table has any number of records.• Each table has named fields:
– For the “packages” table: “name”, “class”, …– Fields can be added or deleted from a table.– Fields have data types: various size strings, integer, …– Fields can have default values.– Fields can be required to have a value in each record.– Some fields can be required to be unique within a table.
• Each record has values for some or all of the fields.
National Computational Science
ODA Interface Requirements and Capabilities
• Database interface for OSCAR clusters
• Use mySQL as the database engine
• Hide mySQL interface details
• Hide mySQL interface versions details
• Hide differing mySQL version capabilities
• Both command line and perl interfaces
• Both detailed and simple (helpful) interfaces
• Allow command line unique abbreviations
• Allow raw mySQL command syntax
National Computational Science
Additional ODA Requirements
• Read-only non-root access
• Cluster wide access to the database
• Common command names for cli and perl
• Common arguments for cli and perl
• Error checking on arguments
• Perl functions set un-supplied arguments to defaults
• Connect/Disconnect to database done automatically
National Computational Science
Common API Requirement
• Commands extendable through defined shortcuts
• Shortcuts can be passed arguments
• Shortcuts can use other shortcuts recursively
• Shortcuts are stored in database
• Shortcuts can include perl fragments
National Computational Science
CLI and Perl Common Arguments
• TABLE – table name
• FIELD – field name, has to be unique in listed tables
• TABLE.FIELD – explicitly named field of named table
• ASSIGNMENT – assigns a field a new value:FIELD~VALUE TABLE.FIELD~VALUE
• MODIFIER – narrows down records output/modifiedFIELD COMPARE VALUE
FIELD COMPARE TABLE.FIELD
TABLE.FIELD COMPARE FIELD
VALUE COMPARE FIELD
• COMPARE OPERATORS: = != > < >= <= etc.
National Computational Science
Examples Using Command Arguments
root> oda read_records packages.class name=pfilter
included
root> oda modify_records packages.name=sis vendor~yoda
root> oda read_records packages vendor=me group
Security
root> oda read_records packages.name class class=core
switcher core
sis core
perl-Qt core
c3 core
base core
National Computational Science
Commonly Used CLI and Perl Module Commands
• add_fields – add one or more fields to a table
• create_shortcut – create a new shortcut
• create_table – create a new table
• delete_records – delete one or more table records
• delete_shortcuts – delete one or more shortcuts
• execute_command – expand and execute a command
• execute_raw_command – execute a command
• insert_record – insert one record into a table
• modify_records – modify one or more table records
• read_records – read one or more table records
National Computational Science
Other CLI and Perl Module Commands
• connect – connect to a database
• create_database – create a new database
• disconnect – disconnect from a database
• drop_database – destroy/delete a database
• drop_tables – destroy/delete one or more tables
• list_databases – list all database names
• list_driver_types – list all database driver types
• list_fields – list the field names in a table
• list_tables – list the tables in a database
National Computational Science
Command Line Only ODA Commands
• dump – pretty printing of database dataroot@system ]# oda dump oscar
################################################
TABLE: oscar
installation_date: <null>
interface: eth0
oscar_server_architecture: ia32
oscar_server_distribution: redhat
oscar_server_distribution_version: 9
oscar_server_node_id: <null>
oscar_version: 2.3
################################################
National Computational Science
oda.pm read_records
# inputs: options reference to options hash
# tables_fields reference to tables/fields list
# wheres reference to where expressions list
# results reference to list or hash for results
# hash_flag non-zero if list of hashes wanted,
# zero if list of lists wanted
# errors_ref reference to error strings list
# number_ref reference to number of records modified
# outputs: non-zero is returned if no errors
# requested fields from each result record are placed
# in a list, references to those lists are placed
# in the results list that caller passed a reference to
National Computational Science
oda.pm modify_records
# inputs: options reference to options hash
# tables pointer to tables list
# assigns pointer to assign strings list
# wheres reference to where expressions list
# hash_flag non-zero if list of hashes wanted,
# zero if list of lists wanted
# errors_ref reference to error strings list
# number_ref reference to number of records modified
# outputs: non-zero is returned if no errors
# errors strings put in errors_ref list
National Computational Science
Database.pm database_read_table_fields
# reads specified fields from all the records in a specified database# table into a double deep hash with the first level of keys being# taken fromt the "name" field for each record, with data in the first# level of keys being pointers to hashes one per database table record,# with the second level hashes being the requested fields with their# data being the values in the database fields.## paramaters are: table database table name# fields pointer to requested field nanes list# (if undef or empty returns all fields)# wheres pointer to where expressions# (if undef all records returned)# index_field name of index field# (if undef "name" is used)# print_err if defined and non-zero, print out# error messages
National Computational Science
Database.pm database_execute_command
# This function executes an oda database command, parsing# the command from one or more string arguments, expanding# any database shortcuts if needed. ## inputs: command_args either a single scalar string that# includes the command/shortcut and any# arguments, or a reference to a list# strings that include the command/shortcut# and any arguments.# results_ref reference to a variable for the results,# commands that return one or more strings# will place a reference to the list of# result strings in results_ref, commands# that do not return any result strings# will place the integer number of records# affected or modified in results_ref# print_errors if defined and non-zero, print out# error messages
National Computational Science
Simple and Derived Database Tables
• Table nodes fields:– id
– name
– …
• Table node_groups fields:– id
– name
• Table node_groups_included_nodes fields:– node_id
– node_group_id
National Computational Science
Defining Tables and Fields in config.xml
<database_tables>
<node_groups>
<fields>
<id>
<type> integer </type>
<parameters> auto_increment not null unique primary key </parameters>
</id>
<name>
<parameters> not null unique </parameters>
</name>
</fields>
</node_groups>
</database_tables>
National Computational Science
Defining Shortcuts in Package config.xml Files
<shortcut> delete_node_group remove_all_nodes_from_node_group %1 ; delete_records node_groups name=%1</shortcut>
<shortcut> remove_all_nodes_from_node_group delete_records node_groups_included_nodes node_groups.name=%1 node_groups.id=node_groups_included_nodes.node_group_id</shortcut>
National Computational Science
Defining Shortcuts on the Command Line
root> oda create_shortcut packages_of_class \
read_records packages.class=%1 %+
root> oda packages_of_class core
switcher
sis
perl-Qt
c3
base
root> oda packages_of_class core vendor=yoda
sis
National Computational Science
OSCAR Simple Tables
• disk_layouts
• disk_partitions
• images
• networks
• nics
• node_groups
• nodes
• packages
National Computational Science
OSCAR Derived Tables
• node_groups_included_nodes
• packages_filters
• packages_provides
• packages_requires
• packages_rpmlists
National Computational Science
For Further ODA Information
• Contact me at:[email protected]
• OSCAR sourceforge project page:– http://sourceforge.net/projects/oscar/