Remove env config

DEVELOPMENT.md

@@ -9,9 +9,9 @@ python setup.py develop
 If you do not plan on modifying the code and will simply be using it, instead run:
 ```
 python setup.py install
-```  
+```
 
-If you have the AWS CLI, you can run `aws configure` to generate the credentials files in the appropriate place. If you have already configured the AWS CLI, then no further steps are necessary. 
+If you have the AWS CLI, you can run `aws configure` to generate the credentials files in the appropriate place. If you have already configured the AWS CLI, then no further steps are necessary.
 
 You must ensure that the account you are authenticating with has at least the following permissions:
 
@@ -20,7 +20,7 @@ You must ensure that the account you are authenticating with has at least the fo
 "ec2:DescribeRegions"], "Effect": "Allow", "Resource": "*" }]}
 ```
 
-This is required to perform the VPC lookups. 
+This is required to perform the VPC lookups.
 
 
 Run/Test/Clean
@@ -40,7 +40,7 @@ python setup.py test
 python setup.py clean —-all
 ```
 
-Note: *.pyc files will be regenerated in src whenever you run the test suite but as long as they are git ignored it’s not a big deal. You can still remove them with `rm src/**/*.pyc` 
+Note: *.pyc files will be regenerated in src whenever you run the test suite but as long as they are git ignored it’s not a big deal. You can still remove them with `rm src/**/*.pyc`
 
 
 Getting started
@@ -61,24 +61,23 @@ class MyEnv(NetworkBase):
         self.add_child_template(Bastion())
 
 if __name__ == '__main__':
-    my_config = EnvConfig(config_handlers=[Bastion])
-    MyEnv(env_config=my_config)
+    MyEnv()
 
 ```
 
 To generate the cloudformation template for this python code, save the above snippet in a file called `my_env.py` and run `python my_env.py init`.
 
-This will look at the patterns passed into the EnvConfig object and generate a config.json file with the relevant fields added. Fill this config file out, adding values for at least the following fields:  
+This will generate a config.json using the default config (and any loaded subclasses of Template which extend the default config) file with the relevant fields added. Fill this config file out, adding values for at least the following fields:
 
-`template : ec2_key_default` - SSH key used to log into your EC2 instances  
-`template : s3_bucket` - S3 bucket used to upload the generated cloudformation templates  
+`template : ec2_key_default` - SSH key used to log into your EC2 instances
+`template : s3_bucket` - S3 bucket used to upload the generated cloudformation templates
 
 Next run `python my_env.py create` to generate the cloudformation template using the updated config. Since we overrode environmentbase's `create_hook` function, this will hook into environmentbase's create action and add the bastion stack and any other resources you specified.
 
-NOTE: You can also override config values using environment variables. You can create env variables using the format:  
-`<section label>_<config_key>` in all caps (e.g. `TEMPLATE_EC2_KEY_DEFAULT`)  
+NOTE: You can also override config values using environment variables. You can create env variables using the format:
+`<section label>_<config_key>` in all caps (e.g. `TEMPLATE_EC2_KEY_DEFAULT`)
 
-These are read in after the config file is loaded, so will override any values in your config.json  
+These are read in after the config file is loaded, so will override any values in your config.json
 
 Then run `python my_env.py deploy` to create the stack on [cloudformation](https://console.aws.amazon.com/cloudformation/)
 
@@ -102,12 +101,12 @@ By default, networkbase will create one public and one private subnet for each a
         ],
         "subnet_config": [
             {
-                "type": "public", 
+                "type": "public",
                 "size": "18",
                 "name": "public"
             },
             {
-                "type": "private", 
+                "type": "private",
                 "size": "22",
                 "name": "private"
             },
@@ -136,18 +135,18 @@ Extension point for modifying behavior of delete action. Called after config is
 Extension point for reacting to the cloudformation stack event stream.  If global.monitor_stack is enabled in config this function is used to react to stack events. Once a stack is created a notification topic will begin emitting events to a queue.  Each event is passed to this call for further processing.  The return value is used to indicate whether processing is complete (true indicates processing is complete, false indicates you are not yet done).
 Details about the event data can be read [here](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-listing-event-history.html)
 
-The event_data hash provided the following mappings from the raw cloudformation event:  
-status = ResourceStatus  
-type = ResourceType  
-name = LogicalResourceId  
-reason = ResourceStatusReason  
-props = ResourceProperties  
+The event_data hash provided the following mappings from the raw cloudformation event:
+status = ResourceStatus
+type = ResourceType
+name = LogicalResourceId
+reason = ResourceStatusReason
+props = ResourceProperties
 
 ## Dealing with versions
 
 1. Basic commands
 
-  Reading tag information 
+  Reading tag information
 
   - List all tags (apply across all branches): `git tag -l`
 
@@ -197,7 +196,7 @@ props = ResourceProperties
   2. Make code changes, test .. etc (merge changes from master into this branch)
      Remember to update: src/environmentbase/version.py to the same value as the branch name.
   3. Merge into master
-  4. Delete the version branch 
+  4. Delete the version branch
   5. Create tag on master with same name as branch you just deleted.
-  
+
   Note that the branch is deleted **before** the tag is created because they share the same name.  Otherwise referring to things by the version name may be ambigious.

src/environmentbase/environmentbase.py

@@ -24,24 +24,16 @@ class ValidationError(Exception):
     pass
 
 
-class EnvConfig(object):
-
-    def __init__(self, config_handlers=None):
-        self.config_handlers = config_handlers if config_handlers else []
-        # self.stack_event_handlers = stack_event_handlers if stack_event_handlers else []
-        # self.deploy_handlers = deploy_handlers if deploy_handlers else {}
-
-
 class EnvironmentBase(object):
     """
     EnvironmentBase encapsulates functionality required to build and deploy a network and common resources for object storage within a specified region
     """
 
     def __init__(self,
                  view=None,
-                 env_config=EnvConfig(),
                  config_filename=res.R.CONFIG_FILENAME,
-                 config_file_override=None):
+                 config_file_override=None,
+                 is_silent=False):
         """
         Init method for environment base creates all common objects for a given environment within the CloudFormation
         template including a network, s3 bucket and requisite policies to allow ELB Access log aggregation and
@@ -53,7 +45,6 @@ def __init__(self,
         """
 
         self.config_filename = config_filename
-        self.env_config = env_config
         self.config_file_override = config_file_override
         self.config = {}
         self.globals = {}
@@ -62,18 +53,16 @@ def __init__(self,
         self.deploy_parameter_bindings = []
         self.ignore_outputs = ['templateValidationHash', 'dateGenerated']
         self.stack_outputs = {}
-        self._config_handlers = []
         self.stack_monitor = None
         self._ami_cache = None
         self.cfn_connection = None
         self.sts_credentials = None
 
         self.boto_session = None
 
-        # self.env_config = env_config
-        for config_handler in env_config.config_handlers:
-            self._add_config_handler(config_handler)
-        self.add_config_hook()
+        # Show names of Template subclasses
+        if not is_silent:
+            print "Using patterns: %s" % [cls.__name__ for cls in utility.get_pattern_list()]
 
         # Load the user interface
         self.view = view if view else cli.CLI()
@@ -92,14 +81,6 @@ def create_hook(self):
         """
         pass
 
-    def add_config_hook(self):
-        """
-        Override in your subclass for adding custom config handlers.
-        Called after the other config handlers have been added.
-        After the hook completes the view is loaded and started.
-        """
-        pass
-
     def deploy_hook(self):
         """
         Extension point for modifying behavior of deploy action. Called after config is loaded and before
@@ -148,8 +129,7 @@ def init_action(self, is_silent=False):
         Override in your subclass for custom initialization steps
         @param is_silent [boolean], supress console output (for testing)
         """
-        config_handlers = self.env_config.config_handlers
-        res.R.generate_config(prompt=True, is_silent=is_silent, output_filename=self.config_filename, config_handlers=config_handlers)
+        res.R.generate_config(prompt=True, is_silent=is_silent, output_filename=self.config_filename)
 
     def s3_prefix(self):
         """
@@ -250,13 +230,30 @@ def _root_template_url(self):
             bucket_name=self.template_args.get('s3_bucket'),
             resource_path=self._root_template_path())
 
+    def load_runtime_config(self):
+        """
+        For patterns defining custom config sections bind the loaded config values to associated class.
+        For class TestPattern whose get_factory_defaults() returns { "fav_color": "red" } will be able
+        to retreive the loaded value from the build_hook() (e.g. TestPattern.runtime_config['fav_color'])
+        """
+        pattern_classes = utility.get_pattern_list()
+        for cls in pattern_classes:
+            runtime_config = {}
+            default_config = cls.get_factory_defaults()
+            for key in default_config.keys():
+                value = self.config[key]
+                runtime_config[key] = value
+
+            cls.runtime_config = runtime_config
+
     def create_action(self):
         """
         Default create_action invoked by the CLI
         Loads and validates config, initializes a new template instance, and writes it to file.
         Override the create_hook in your environment to inject all of your cloudformation resources
         """
         self.load_config()
+        self.load_runtime_config()
         self.initialize_template()
 
         # Do custom troposphere resource creation in your overridden copy of this method
@@ -461,27 +458,13 @@ def _validate_config(self, config, factory_schema=None):
         config_reqs_copy = copy.deepcopy(factory_schema)
 
         # Merge in any requirements provided by config handlers
-        for handler in self._config_handlers:
-            config_reqs_copy.update(handler.get_config_schema())
+        utility.update_schema_from_patterns(config_reqs_copy)
 
         self._validate_config_helper(config_reqs_copy, config, '')
 
         # Validate region
         self._validate_region(config)
 
-    def _add_config_handler(self, handler):
-        """
-        Register classes that will augment the configuration defaults and/or validation logic here
-        """
-
-        if not hasattr(handler, 'get_factory_defaults') or not callable(getattr(handler, 'get_factory_defaults')):
-            raise ValidationError('Class %s cannot be a config handler, missing get_factory_defaults()' % type(handler).__name__ )
-
-        if not hasattr(handler, 'get_config_schema') or not callable(getattr(handler, 'get_config_schema')):
-            raise ValidationError('Class %s cannot be a config handler, missing get_config_schema()' % type(handler).__name__ )
-
-        self._config_handlers.append(handler)
-
     @staticmethod
     def _config_env_override(config, path, print_debug=False):
         """

src/environmentbase/networkbase.py

@@ -10,19 +10,10 @@ class NetworkBase(EnvironmentBase):
     for a common deployment within AWS. This is intended to be the 'base' stack for deploying child stacks
     """
 
-    def add_config_hook(self):
-        super(NetworkBase, self).add_config_hook()
-        self._add_config_handler(BaseNetwork)
-
     def create_hook(self):
         super(NetworkBase, self).create_hook()
 
-        network_config = self.config.get('network', {})
-        boto_config = self.config.get('boto', {})
-        nat_config = self.config.get('nat')
-        region_name = boto_config['region_name']
-
-        base_network_template = BaseNetwork('BaseNetwork', network_config, region_name, nat_config)
+        base_network_template = BaseNetwork('BaseNetwork')
         self.add_child_template(base_network_template)
 
         self.template._subnets = base_network_template._subnets.copy()

src/environmentbase/patterns/base_network.py

@@ -73,13 +73,12 @@ def get_factory_defaults():
     def get_config_schema():
         return BaseNetwork.CONFIG_SCHEMA
 
-    def __init__(self, template_name, network_config, region_name, nat_config):
+    def __init__(self, template_name):
         super(BaseNetwork, self).__init__(template_name)
 
-        self.network_config = network_config
-        self.nat_config = nat_config
-        self.az_count = int(network_config.get('az_count', '2'))
-        self.region_name = region_name
+        self.network_config = self.runtime_config['network']
+        self.nat_config = self.runtime_config['nat']
+        self.az_count = int(self.network_config.get('az_count', '2'))
         self.stack_outputs = {}
 
         # Simple mapping of AZs to NATs, to prevent creating duplicates

src/environmentbase/patterns/bastion.py

@@ -9,18 +9,12 @@ class Bastion(Template):
     Adds a bastion host within a given deployment based on environemntbase.
     """
 
-    SUGGESTED_INSTANCE_TYPES = [
-        "m1.small", "t2.micro", "t2.small", "t2.medium",
-        "m3.medium",
-        "c3.large", "c3.2xlarge"
-    ]
-
     def __init__(self,
                  name='bastion',
-                 ingress_port='2222',
-                 access_cidr='0.0.0.0/0',
-                 default_instance_type='t2.micro',
-                 suggested_instance_types=SUGGESTED_INSTANCE_TYPES,
+                 ingress_port=None,
+                 access_cidr=None,
+                 default_instance_type=None,
+                 suggested_instance_types=None,
                  user_data=None):
         """
         Method initializes bastion host in a given environment deployment
@@ -29,15 +23,20 @@ def __init__(self,
         More info here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-elb-listener.html
         @param access_cidr [string] - CIDR notation for external access to this tier.
         @param user_data [string] - User data to to initialize the bastion hosts.
+        @param default_instance_type [string] EC2 instance type name
+        @param suggested_instance_types [list<string>] List of EC2 instance types available for selection in CloudFormation
         """
 
         self.name = name
-        self.ingress_port = ingress_port
-        self.access_cidr = access_cidr
-        self.default_instance_type = default_instance_type
-        self.suggested_instance_types = suggested_instance_types
         self.user_data = user_data
 
+        # Let constructor parameters override runtime config settings
+        cfg = self.runtime_config['bastion']
+        self.ingress_port = ingress_port or cfg['ingress_port']
+        self.access_cidr = access_cidr or cfg['remote_access_cidr']
+        self.default_instance_type = default_instance_type or cfg['default_instance_type']
+        self.suggested_instance_types = suggested_instance_types or cfg['suggested_instance_types']
+
         super(Bastion, self).__init__(template_name=name)
 
     # Called after add_child_template() has attached common parameters and some instance attributes:
@@ -72,7 +71,7 @@ def build_hook(self):
             utility_bucket=self.utility_bucket
         )
 
-        bastion_asg = self.add_asg(
+        self.add_asg(
             layer_name=self.name,
             security_groups=[security_groups['bastion'], self.common_security_group],
             load_balancer=bastion_elb,
@@ -88,7 +87,7 @@ def build_hook(self):
 
         self.add_output(Output(
             'BastionELBDNSZoneId',
-             Value=GetAtt(bastion_elb, 'CanonicalHostedZoneNameID')
+            Value=GetAtt(bastion_elb, 'CanonicalHostedZoneNameID')
          ))
 
         self.add_output(Output(
@@ -99,15 +98,21 @@ def build_hook(self):
     @staticmethod
     def get_factory_defaults():
         return {"bastion": {
-            "instance_type": "t2.micro",
+            "default_instance_type": "t2.micro",
+            "suggested_instance_types": [
+                "m1.small", "t2.micro", "t2.small", "t2.medium",
+                "m3.medium",
+                "c3.large", "c3.2xlarge"
+            ],
             "remote_access_cidr": "0.0.0.0/0",
             "ingress_port": 2222
         }}
 
     @staticmethod
     def get_config_schema():
         return {"bastion": {
-            "instance_type": "str",
+            "default_instance_type": "str",
+            "suggested_instance_types": "list",
             "remote_access_cidr": "str",
             "ingress_port": "int"
         }}