Add controllers for roles users sindex udf (#108)

* Create manage acl controller * Add manage udfs * Change tests to use a server with security enabled to accomidate manage acl testing * Create tests for manage acl * create "show user/roles" in sheets * Create `show users` and `show roles` e2e tests * change -r help info * Add `show udfs` command * Add show/manage udf tests * add manage udf help info * create `manage sindex` controller * create `manage sindex` tests * fix collectinfo `show config xdr` error * TOOLS-1627 Fix histogram -f option not working in py3 * Add show sindex controller to default and collectinfo analyzer * Add enable/disable to access asinfo and manage. Add show users, roles, udfs, and sindex to collectinfo-analyzer. * Add enable --warn flag for manage commands * Tools-1638 Fix `info xdr` in collectinfo-analyzer for server < 5.0 * TOOLS-1643 Fix traceback for 'show config xdr' and 'show config dc' * TOOLS-1632 Too many sprigs per partition rule should be for all-flash * [TOOLS-1643] Add warning message when there is an error parsing xdr configs * TOOLS-1634 Fix commit bug * TOOLS-1636 "Show distribution" shows same values for every column in a row * TOOLS-1649 fix log analyzer 'histogram' average output * TOOLS-1652 Remove 'src-id' from info xdr title * Add action to upload codecoverage to code cov Co-authored-by: Kevin Porter <[email protected]>
aerospike · Feb 17, 2021 · 2f75dc3 · 2f75dc3
2 parents b945a73 + cfc4a5c
commit 2f75dc3
Show file tree

Hide file tree

Showing 46 changed files with 3,393 additions and 560 deletions.
diff --git a/.coveragerc b/.coveragerc
@@ -0,0 +1,4 @@
+[run]
+
+include = 
+    lib/*
diff --git a/.github/workflows/asadm-codecov.yml b/.github/workflows/asadm-codecov.yml
@@ -0,0 +1,31 @@
+name: Codecov
+on:
+  pull_request:
+    branches:
+      - master
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.9]
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install flake8 unittest2 coverage mock
+        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+    - name: Test with unittest2
+      run: |
+        coverage run -m unittest2 discover -s test/unit -t .
+    - name: Upload coverage to Codecov
+      uses: codecov/codecov-action@v1
+      with:
+        token: ${{secrets.CODECOV_TOKEN}}
+        flags: unittests
+        verbose: false
diff --git a/README.md b/README.md
@@ -1,7 +1,8 @@
 # Aerospike Admin
 ## Description
 Aerospike Admin provides an interface for Aerospike users to view the stat
-of their Aerospike Cluster by fetching information from running cluster (Cluster mode) or logs (Log-analyser mode).
+of their Aerospike Cluster by fetching information from a running cluster (Cluster mode) 
+a collectinfo file (Collectinfo-Analyzer), or logs (Log-analyser mode).
 Start the tool with *asadm* and run the *help* command to get started.
 
 ## Installing Aerospike Admin
@@ -66,8 +67,10 @@ sudo easy_install -a readline
 
 ### Setting Test Environment
 asadm has unit and e2e tests. To setup environment for e2e tests, execute following steps:
+- Enable security in the aerospike.conf file.
+- Verify that the default user `admin` exists and that is has the default roles: `sys-admin`, `user-admin`, and `read-write`. 
 - Start Aerospike cluster: Test machine should be part of this cluster with 3000 as asinfo port.
-- Write few records to cluster
+- Write few records to cluster `asbenchmark -h <host> -Uadmin -Padmin`
 - Wait for few seconds so cluster can return histogram output properly.
 
 ### Running Tests

diff --git a/asadm.py b/asadm.py
@@ -107,7 +107,10 @@ def critical(self, msg, *args, **kwargs):
 CMD_FILE_MULTI_LINE_COMMENT_START = "/*"
 CMD_FILE_MULTI_LINE_COMMENT_END = "*/"
 
-MULTILEVEL_COMMANDS = ["show", "info"]
+MULTILEVEL_COMMANDS = ["show", "info", "manage"]
+
+DEFAULT_PROMPT = "Admin> "
+PRIVILEGED_PROMPT = "Admin+> "
 
 
 class AerospikeShell(cmd.Cmd):
@@ -119,6 +122,7 @@ def __init__(self, admin_version, seeds, user=None, password=None, auth_mode=Aut
         self.connected  = True
         self.admin_history = ADMIN_HOME + 'admin_' + str(mode).lower() + "_history"
         self.execute_only_mode = execute_only_mode
+        self.privileged_mode = False
 
         if mode == AdminMode.LOG_ANALYZER:
             self.name = 'Aerospike Log Analyzer Shell'
@@ -152,7 +156,7 @@ def __init__(self, admin_version, seeds, user=None, password=None, auth_mode=Aut
                                                       clinfo_path=log_path)
                 self.prompt = "Collectinfo-analyzer> "
                 if not execute_only_mode:
-                    self.intro = str(self.ctrl.loghdlr)
+                    self.intro = str(self.ctrl.log_handler)
 
             else:
                 if user is not None:
@@ -180,7 +184,7 @@ def __init__(self, admin_version, seeds, user=None, password=None, auth_mode=Aut
                     else:
                         logger.critical("Not able to connect any cluster with " + str(seeds) + ".")
 
-                self.prompt = "Admin> "
+                self.set_prompt(DEFAULT_PROMPT)
                 self.intro = ""
                 if not execute_only_mode:
                     self.intro += str(self.ctrl.cluster) + "\n"
@@ -196,10 +200,6 @@ def __init__(self, admin_version, seeds, user=None, password=None, auth_mode=Aut
                         self.intro += terminal.fg_red() + "Extra nodes in alumni list: %s" % (
                             ", ".join(cluster_down_nodes)) + terminal.fg_clear() + "\n"
 
-            if self.use_rawinput:
-                self.prompt = "\001" + terminal.bold() + terminal.fg_red() + "\002" +\
-                              self.prompt + "\001" +\
-                              terminal.unbold() + terminal.fg_clear() + "\002"
         except Exception as e:
             self.do_exit('')
             logger.critical(str(e))
@@ -220,6 +220,14 @@ def __init__(self, admin_version, seeds, user=None, password=None, auth_mode=Aut
             if command != 'help':
                 self.commands.add(command)
 
+    def set_prompt(self, prompt):
+        self.prompt = prompt
+
+        if self.use_rawinput:
+            self.prompt = "\001" + terminal.bold() + terminal.fg_red() + "\002" +\
+                                        self.prompt + "\001" +\
+                                        terminal.unbold() + terminal.fg_clear() + "\002"
+
     def clean_line(self, line):
         # get rid of extra whitespace
         lexer = shlex.shlex(line)
@@ -269,6 +277,7 @@ def precmd(self, line, max_commands_to_print_header=1,
                 return " ".join(line)
 
             if len(lines) > max_commands_to_print_header:
+
                 if len(line) > 1 and any(cmd.startswith(line[0]) for cmd in MULTILEVEL_COMMANDS):
                     index = command_index_to_print_from
                 else:
@@ -279,10 +288,19 @@ def precmd(self, line, max_commands_to_print_header=1,
                     terminal.bold(), ' '.join(line[index:]), terminal.reset()))
 
             sys.stdout.write(terminal.reset())
+
             try:
                 response = self.ctrl.execute(line)
+
                 if response == "EXIT":
                     return "exit"
+
+                elif response == "ENABLE":
+                    self.set_prompt(PRIVILEGED_PROMPT)
+
+                elif response == "DISABLE":
+                    self.set_prompt(DEFAULT_PROMPT)
+
             except Exception as e:
                 logger.error(e)
         return ""  # line was handled by execute
@@ -330,48 +348,41 @@ def complete_path(self, args):
         return self._complete_path(args[-1])
 
     def completenames(self, text, line, begidx, endidx):
-        try:
-            origline = line
+        origline = line
 
-            if isinstance(origline, str):
-                line = origline.split(" ")
-                line = [v for v in map(str.strip, line) if v]
-                if origline and origline[-1] == ' ':
-                    line.append('')
+        if isinstance(origline, str):
+            line = origline.split(" ")
+            line = [v for v in map(str.strip, line) if v]
+            if origline and origline[-1] == ' ':
+                line.append('')
 
-            if len(line) > 0:
-                self.ctrl._init_commands()  # dirty
-                cmds = self.ctrl.commands.get_key(line[0])
-            else:
-                cmds = []
-
-            watch = False
-            if len(cmds) == 1:
-                cmd = cmds[0]
-                if cmd == 'help':
-                    line.pop(0)
-                if cmd == 'watch':
-                    watch = True
-                    line.pop(0)
-                    try:
-                        for _ in (1, 2):
-                            int(line[0])
-                            line.pop(0)
-                    except Exception:
-                        pass
-            line_copy = copy.deepcopy(line)
-            names = self.ctrl.complete(line)
-            if watch:
+        if len(line) > 0:
+            self.ctrl._init_commands()  # dirty
+            cmds = self.ctrl.commands.get_key(line[0])
+        else:
+            cmds = []
+
+        watch = False
+        if len(cmds) == 1:
+            cmd = cmds[0]
+            if cmd == 'help':
+                line.pop(0)
+            if cmd == 'watch':
+                watch = True
+                line.pop(0)
                 try:
-                    names.remove('watch')
+                    for _ in (1, 2):
+                        int(line[0])
+                        line.pop(0)
                 except Exception:
                     pass
-            if not names:
-                names = self.complete_path(line_copy) + [None]
-                return names
+        line_copy = copy.deepcopy(line)
+        names = self.ctrl.complete(line)
+
+        if not names:
+            names = self.complete_path(line_copy) + [None]
+            return names
 
-        except Exception:
-            return []
         return ["%s " % n for n in names]
 
     def complete(self, text, state):
@@ -380,17 +391,15 @@ def complete(self, text, state):
         If a command has not been entered, then complete against command list.
         Otherwise try to call complete_<command> to get list of completions.
         """
-        try:
-            if state >= 0:
-                origline = readline.get_line_buffer()
-                line = origline.lstrip()
-                stripped = len(origline) - len(line)
-                begidx = readline.get_begidx() - stripped
-                endidx = readline.get_endidx() - stripped
-                compfunc = self.completenames
-                self.completion_matches = compfunc(text, line, begidx, endidx)
-        except Exception:
-            pass
+
+        if state >= 0:
+            origline = readline.get_line_buffer()
+            line = origline.lstrip()
+            stripped = len(origline) - len(line)
+            begidx = readline.get_begidx() - stripped
+            endidx = readline.get_endidx() - stripped
+            compfunc = self.completenames
+            self.completion_matches = compfunc(text, line, begidx, endidx)
 
         try:
             return self.completion_matches[state]
@@ -592,7 +601,7 @@ def main():
     if cli_args.asinfo_mode:
 
         if mode == AdminMode.COLLECTINFO_ANALYZER or mode == AdminMode.LOG_ANALYZER:
-            logger.critical("asinfo mode can not work with Collectinfo-analyser or Log-analyser mode.")
+            logger.critical("asinfo mode cannot work with Collectinfo-analyser or Log-analyser mode.")
 
         commands_arg = cli_args.execute
         if commands_arg and os.path.isfile(commands_arg):

diff --git a/doc/asadmin.md b/doc/asadmin.md
@@ -47,7 +47,7 @@
 ##### info xdr
 - Same as current except always sorted by **alias**.
 
-#### clinfo/asinfo
+#### asinfo
 ##### Modifiers
 - with
 
@@ -71,7 +71,7 @@
     - **xdr** - show statistics for xdr
     - **set** - show statistics for set
 
-##### latency
+##### latencies
 - **latency** - Show Aerospike latency information sorted by **alias**.
 
 ##### show config
@@ -87,44 +87,9 @@
 ###### Modifier
 - **diff** - only show parameters that are different for the nodes selected.
     - IE =show config xdr compare= would only show parameters that are different.
-
-#### set
-- The purpose of set is to provide an easier interface to set dynamic options as well as allow tab completion for the various options.
-
-##### Modifiers
-- **with**
-
-##### Default
-- Show help
-
-##### set service
-- **service** <config name> <value>
-
-##### set network.heartbeat
-- **network.heartbeat** <config name> <value>
-
-##### set network.info
-- **network.info** <config name> <value>
-
-##### set namespace
-- **namespace** <namespace name> <config name> <value>
-- **namespace** <namespace name> <set name> <config name> <value>
-
-##### set xdr
-- **xdr** <config name> <value>
-
-#### exec
-##### Modifiers
-- with
-
-##### TBD
-- Requires SSH
-- Execute a shell command on node selection
-
 #### watch
 - Similar to current watch
 - Default to display every 10 seconds and only if there are changes
-- Would be nice to highlight changes (similar to watch -d)
 - {,#} - if a number is not provided, watch will check for changes every
 	      10 seconds otherwise every provided number seconds. This command
 	      may be used to prefix any other command.
@@ -140,12 +105,15 @@
 ## Developer Guide
 Important files and structure:
 
-1. /asadmin.py <br>
+1. /asadmin.py
+
    asadmin.py is the entry point specifically the **precmd** performs a
    search and finds the actual command the user is requesting and executes.
    <br>
    For most updates this file will not need modified.
-2. /lib/controller.py <br>
+
+2. /lib/controller.py
+
    The controller is where commands are defined. Each command has a
    **commandHelp** decorator that accepts a list of lines to be displayed when
    help on a command is requested.
@@ -154,26 +122,38 @@ Important files and structure:
    **RootController**. End points in the hierarchy will be methods of a controller
    that are prefixed "do_", default controller behavior will be prefixed
    "\_do\_".
-3. /lib/view.py <br>
+
+3. /lib/view/view.py
+
    With a little exception, nothing prints unless it is defined in view.py.
    This is where the results are rendered to the user. Very likely if you are
    adding a feature you will need to add code here.
-4. /lib/table.py <br>
+
+4. /lib/view/table.py  Replaced by sheets.py
+
    The table class handles presentation of info and show commands and may work
    for yours as well.
    In the table module, the Extractors class defines various numeric formaters
    for instance if you wanted to display uptime in hrs/days use timeExtractor.
-   <br>
+
    Before adding rows to the table you need to define a list of column names,
    if a column is being renamed, use a tuple of ('original\_name', 'new\_name')
-   <br>
+
    A datasource is really a data transformation, if you want to use
    timeExtractor on a column then you would need to call **addDataSource** on a
    new column name and pass the extractor as the function to do the
    transformation, passing the function the old columnname.
 
-   Afterward will need to add data to the table one row at a time.
-5. /lib/cluster.py <br>
+5. /lib/view/sheets.py
+
+   The sheet class was made as a retrospective replacement of table.py.  It was
+   mostly developed by Kevin Porter.  Creating a table using sheets requires that
+   you create a template defining how your table should appear, where it should
+   extract (project) data from, if it should be converted, and if it should be 
+   aggregated.  You can see all the created templates in /lib/view/templates.py
+
+5. /lib/cluster.py
+
    Calls node methods in parallel, shouldn't need to modify for anything other
    than bug fixes.