Refactor Analtyical methods and improve imports (#189)

* Initial push of SAS Viya procedures

* Add to_json method to the SAS data object.

* update doc, wrap function for correct docstring, add TODO's

* convert reg method

* convert reg method

* update mixed method (proc mixed)

* add functionality to convert model syntax to input target syntax
other general cleanup

* emit warning message if python version is older than 3.4

* minor cleanup

* allow var statement to take str and list

* merge changes in sasproccommons and convert glm and mixed methods in sasstat

* improve debug messages and target/input handling

* add additional logging

* move proc decorator to new file for use across all procs.

* fix imports and loggers

* add the SAS Viya ML procedures as methods for the viyaml object

* improve decorator function to generate doc for procedure methods

* replace the stat methods with a new decorator to make tab complete and documentation easier. Should not break existing code

* convert sasutil to use proc_decorator

* convert sasets to use proc_decorator

* convert sasqc to use proc_decorator
add statements for SHEWHART to sasproccommons.

* update doc links for sasstat

* convert sasml to use proc_decorator

* convert sasViyaML to use proc_decorator

* clean up imports and format code

* add savestate for strings and reformat file.

* Change imports to be more specific no more '*'

* create Codegen class to simplify code generation for SAS statements

* Fix imports to avoid import errors

* initial push of test cases for ML and ViyaML

* remove commented code blocks from sasioiom and update imports for sasbase

* fix imports and continue moving code into codegen class

* change imports and finish moving code into codegen class

* add back commented code for potential future work.

* update imports and minor code changes

* ensure order of proc statements in some cases to avoid errors (input before target, class before model)

* simplify the assignment of libref in pandas df creation

* add boolean types for several statements proper name of hpclus

* update and expand test cases

* turn off debug and update docstrings

* fix issues with codegen based on testware.

* create smoke tests for ViyaML

* update signatures and docstrings based on accepted types

* change test assertion for UCM tests because the number of ODS tables and graphs has changed

* change import to include SASResults

* fix codegen issues related to refactor

* change logging level to WARN everywhere

* minor changes to pandas tests and code

* minor changes to testware and code to fix issues.

* reformat test_sasdata

* change __init__ imports
reorder imports for sasiostdio
comment out debug import for sasbase

* fix testcase missing required parameter. update doc

* reformat for pep8

* create testware for ViyaML and minor fixes in codegen to handle new code base.

* changes for ViyaML

* Update the documentation for how to add a procedure as a new method.

* add methods for issue #183.
in sasets methods: autoreg, expand, varmax, x11, x12
in sasutil method: univariate

* Merge in changes from Tom's review. Mostly fix imports and revert formatting.

* Remove new methods from issue 183

* make output and out compatible

* add support for 'out' statement to match 'output'. Support output types of str, bool, SASdata
The out statement was used prior to 2.4.0 but the statement for most procedures is 'output' both are now treated correctly.

* handle output statements that include 'out=' str or just str

saspy/__init__.py

@@ -13,17 +13,17 @@
 #  See the License for the specific language governing permissions and
 #  limitations under the License.
 #
-from saspy.sasbase        import *
-from saspy.sasstat        import *
-from saspy.sasets         import *
-from saspy.sasml          import *
-from saspy.sasqc          import *
-from saspy.sasresults     import *
-from saspy.sasutil        import *
-from saspy.sastabulate    import *
-from saspy.sasproccommons import *
-from saspy.SASLogLexer    import *
-from saspy.version        import __version__
+
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from saspy.version import __version__
+from saspy.sasbase import SASsession, SASconfig
+from saspy.sasdata import SASdata
+from saspy.sasproccommons import SASProcCommons
+from saspy.sastabulate import Tabulate
+from saspy.sasresults import SASresults
 
 def isnotebook():
     try:
@@ -38,5 +38,5 @@ def isnotebook():
         return False      # Probably standard Python interpreter
 
 if isnotebook():
-	from .sas_magic import SASMagic
-	get_ipython().register_magics(SASMagic)
+    from saspy.sas_magic import SASMagic
+    get_ipython().register_magics(SASMagic)

saspy/doc/source/adding-procedures.rst

@@ -28,25 +28,50 @@ Process
 
 To add a new procedure follow these steps:
 
-#. Create a new method for the procedure.
-#. Create the set of required statements. If there are no required statements 
-   then create an empty set {}.
-#. Create the legal set of statements. This can often be obtained from the 
-   documentation for the procedure. procopts should always be included in the 
-   legal set to permit flexibility in calling the procedure.
-#. Create the doc string with the following parts at a minimum:
-
-    - Procedure name
-    - Required set
-    - Legal set
-    - Link to the procedure documentation
-
-#. Add the return call for the method using an existing procedure as an example.
-#. Verify that all the statements in the required and legal sets are listed in 
-   _makeProcCallMacro method of sasproccommons.py.
-#. Write at least one test to exercise the procedures and include it in the 
+#. Identify the product of the procedure (SAS/STAT, SAS/ETS, SAS Enterprise Miner, etc).
+#. Find the corresponding file in saspy sasstat.py, sasets.py, sasml.py, etc.
+#. Create a set of valid statements. Here is an example:
+
+    .. code-block::
+
+        lset = {'ARIMA', 'BY', 'ID', 'MACURVES', 'MONTHLY', 'OUTPUT', 'VAR'}
+
+    The case and order of the items will be formated.
+#. Call the `doc_convert` method to generate then method call as well as the docstring markup
+
+    .. code-block::
+
+        import saspy
+        print(saspy.sasdecorator.procDecorator.doc_convert(lset, 'x11')['method_stmt'])
+        print(saspy.sasdecorator.procDecorator.doc_convert(lset, 'x11')['markup_stmt'])
+
+
+    The `doc_convert` method takes two arguments: a list of the valid statements and the proc name. It returns a dictionary with two keys, method_stmt and markup_stmt. These outputs can be copied into the appropriate product file.
+
+#. Add the proc decorator to the new method.
+    The decorator should be on the line above the method declaration.
+    The decorator takes one argument, the required statements for the procedure. If there are no required statements than an empty list `{}` should be passed.
+    Here are two examples one with no required arguments:
+
+    .. code-block::
+
+        @procDecorator.proc_decorator({})
+        def esm(self, data: 'SASdata' = None, ...
+
+    And one with required arguments:
+
+    .. code-block::
+
+        @procDecorator.proc_decorator({'model'})
+        def mixed(self, data: 'SASdata' = None, ...
+
+#. Add a link to the SAS documentation plus any additional details will be helpful to users
+
+#. Write at least one test to exercise the procedures and include it in the
    appropriate testing file.
 
+If you have questions, please open an issue in the GitHub repo and the maintainers will be happy to help.
+
 .. Example
 .. =======
 .. Following the procedure above, I will add a method for the ADAPTIVEREG procedure.