Database model reference

 

Here you can find details about database models in CoreCluster.

Model structure

Since 16.09 all models are defined in app.py file missing article .

All model-related files in CoreCluster's installation are keep in models/ directory. This contains __init__.py file, which loads enabled in /etc/corecluster/config.py models and common_models.py which contains definitions of basic DB classes (described below).

The CoreCluster's basic model - core - is stored in subdirectory: models/core. It contains definitions of virtual machines, images, storages etc.

All models enabled in CoreCluster should be listed in /etc/corecluster/config.py file. Each time, when CoreCluster is starting, this list is used to load proper definitions of database tables (and to migrate them):


LOAD_MODELS = ['corecluster.models.core']

Each entry is full module path to be imported by CoreCluster. Until the Core model is placed directly in models/core, all other models could be placed in theirs own modules. For example, installation with CoreTalk should have following LOAD_MODELS entry:


LOAD_MODELS = ['corecluster.models.core',
               'coretalk.models.coretalk',
]

Above list is used for:

CoreModel and Mixins

Each model class in CoreCluster bases on CoreModel class. This allows to keep order in whole application and keep all models simple and extensible. There are two main purposes for CoreModel:

You can import CoreModel with python import:


from corecluster.models.common_models import CoreModel

and use it as base class:


class MyNewModel(CoreModel):
    def __init___(self, *args, **kwargs):
        super(MyNewModel, self).__init__(*args, **kwargs)
        ...

    my_field = models.CharField(max_length=123)
    ...

In common_models (file is located in models/common_models.py) you can also find two additional classes: UserMixin and StateMixin. This two classes adds support for owner of object (user foreign field) and object's state. All methods in above classes will be described in this document.

Inheritance order and __init__ methods

While inheriting CoreModel and any Mixins, you should remember to keep proper order of classes listed in your new class header. The Mixin classes should be listed as first, and CoreModel as the last class:


class MyNewModel(UserMixin, StateMixin, CoreModel):
    ...

CoreModel methods and fields

Data

CoreModel class implements the data field and get/set_prop methods. This allow any extensions to use existing models without modyfing database schema. Also this makes it extensible. While it is not best practice to use "all purpose" fields in databases, this gives opportunity to make code and database models ready for any additional data, not deigned in original core database model (vms, images, etc.). Data in _data field is stored in JSON format. Use ONLY following methods to manage values in data field:

remember - do not use _data field directly, from your model. Instead, use above functions. The good maneer is also to keep naming convention - add prefix with your extension (or model) before all keys stored in any _data field by your extension. With this you should avoid conflicts with other keys from other modules.

In next section you will learn how to serialize data. CoreModel provides the to_dict method, which returns values of the _data field as python dictionary to serialize them.

Serializing data - to_dict

In most API functions you will need to serialize model's data to send it to client. All API functions could return python dictionary with data - serialization is done at lower levels. But still it is necessary to serialize your models' data into python dictionary.

To make it simple CoreModel provides default mechanism: list all your serializable fields in serializable list, in your model class. The CoreModel's property to_dict will use this list to create python dictionary with proper values of fields, directly from your model.


class MyModel(CoreModel):
    def __init__(self, *args, **kwargs):
        super(CoreModel, self).__init__(*args, **kwargs)

    my_first_field = models.IntegerField()
    my_second_field = models.DateTimeField()

    serializable = ['my_first_field', 'my_second_field', 'data', 'id']

As you can see, in above example both fields were listed as serializable. Additional data field is inherited from CoreModel and contains additional data (could be used by other extensions) and id (delivered from CoreModel).

By calling to_dict function, you will get serialized data to python dictionary from your model.

Serializing special data - by dedicated methods

Sometimes it is neccessary to serialize model's data in more sophisticated way. For example convert height into "tall" or "little" categories. Define your new method to serialize data:


class MyModel(CoreModel):
    def _named_height(self):
        if self.height < 150:
            return 'little'
        else:
            return 'tall'

    height = models.IntegerField()

then put it into serializable list as sub-list with name and function to generate data:


    serializable = ['height', ['am_i_high', '_named_height']]

In above example the to_dict method returns dict with fields:

Get method

The next method, which CoreModel delivers is get(object_id) (do not confuse with get_prop). It returns object with given id or raises proper error in case when object doesn't exist:


...
my_user = User.get(vm_id)

If model inherits UserMixin, the get method gets also user_id parameter:


...
my_vm = VM.get(user_id, vm_id)

this is especially useful, when you got context object in API function:


def do_something_with_vm(context, vm_id):
    vm = VM.get(context.user_id, vm_id)

You don't have to care about retreiving it from database and permissions. UserMixin implements access and group related fields, which are used to determine proper permissions.

Note: If you are not using get from UserMixin, you should implement authorization mechanisms (related to objects) at your own. The best example are edit and delete API functions. This functions don't allow to delete objects not owned by user (e.g. with group or public access)
Editing models

All models could be edited by hand in api functions or in agents, by editing particular fields. The other way to implement edit function (e.g. in API function) is to use editable list and edit method form CoreModel. Each model, which should be editable by user should use this mechanism.

Note: In general this is related to models inheriting from UserMixin. Models like Storage, Node etc. should not be editable by user due to security issues.

Puting list of editable fields in model allows to protect whole database by unprivileged editing.


class MyModel(CoreModel):

    my_first_field = models.IntegerField()
    my_second_field = models.DateTimeField()

    serializable = ['my_first_field', 'my_second_field', 'data', 'id']
    editable = ['my_first_field']

In above example method edit inherited from CoreModel allows only to edit my_first_field. Editing any other will raise exception. To use this mechanism in API views, create function like this:


@api_log(log=True)
def edit(context, vm_id, **kwargs):
    """ Edit VM properties """
    vm = VM.get(context.user_id, vm_id)
    vm.edit(**kwargs)
    vm.save()

Above function gets id of edited virtual machine and user id (this is generated by decorator and based on login+password hash or token). The **kwargs dictionary contains dictionary with all editable fields, which are passed directly to edit method of VM model. In case when user wants to edit foribden field, the proper exception is raised.

Validating editable fields

Some values require validation. To do this, place in editable list new sublist with name of field and lambda function to validate it. Lambda function should return true (data is valid) or false (data is not valid).

Following example comes from Image model:


    editable = ['name',
                'description',
                ['type', lambda x: x in Image.image_types],
                ['disk_controller', lambda x: x in hardware.disk_controllers],
                ['video_device', lambda x: x in hardware.video_devices],
                ['network_device', lambda x: x in hardware.network_devices],
                ['access', lambda x: x in Image.image_access],
               ]
Describe model

Another function is describe_model. This returns list of all editable and serializable fields (only names). This is usually used in API to get description of data, which is returned by functions in particular category. This has no other usage (probably) in your application.

UserMixin

Inheriting UserMixin provides new user field in model. This also adds new get method, which accepts user_id and object_id parameters. In case if user doesn't own object, the proper exception is raised. UserMixin implements also access control for group and public objects, by adding proper fields - access and group.

StateMixin

StateMixin adds new state field into model. The following new functions are also added by this mixin:

Defining possible object states

Each model, inheriting StateMixin should has defined list of possible states, which could be assigned to state field by set_state. This is done by overriding the states list. Also there is possible to set default state by overriding default_state (string with value).

< Go back     Author: Maciej Nabozny Published: Feb. 5, 2017, 11:16 p.m.