Method Documentation¶

single¶

manager_utils.manager_utils.single(queryset)¶

Assumes that this model only has one element in the table and returns it. If the table has more than one or no value, an exception is raised.

Returns:	The only model object in the queryset.
Raises:	`DoesNotExist` error when the object does not exist or a `MultipleObjectsReturned` error when thereis more than one object.

Examples:

TestModel.objects.create(int_field=1)
model_obj = single(TestModel.objects)
print(model_obj.int_field)
1

get_or_none¶

manager_utils.manager_utils.get_or_none(queryset, **query_params)¶

Get an object or return None if it doesn’t exist.

Parameters:	query_params – The query parameters used in the lookup.
Returns:	A model object if one exists with the query params, None otherwise.

Examples:

model_obj = get_or_none(TestModel.objects, int_field=1)
print(model_obj)
None

TestModel.objects.create(int_field=1)
model_obj = get_or_none(TestModel.objects, int_field=1)
print(model_obj.int_field)
1

upsert¶

manager_utils.manager_utils.upsert(manager, defaults=None, updates=None, **kwargs)¶

Performs an update on an object or an insert if the object does not exist.

Parameters:

defaults (dict) – These values are set when the object is created, but are irrelevant when the object already exists. This field should only be used when values only need to be set during creation.
updates (dict) – These values are updated when the object is updated. They also override any values provided in the defaults when inserting the object.
kwargs – These values provide the arguments used when checking for the existence of the object. They are used in a similar manner to Django’s get_or_create function.

Returns:

A tuple of the upserted object and a Boolean that is True if it was created (False otherwise)

Examples:

# Upsert a test model with an int value of 1. Use default values that will be given to it when created
model_obj, created = upsert(TestModel.objects, int_field=1, defaults={'float_field': 2.0})
print(created)
True
print(model_obj.int_field, model_obj.float_field)
1, 2.0

# Do an upsert on that same model with different default fields. Since it already exists, the defaults
# are not used
model_obj, created = upsert(TestModel.objects, int_field=1, defaults={'float_field': 3.0})
print(created)
False
print(model_obj.int_field, model_obj.float_field)
1, 2.0

# In order to update the float field in an existing object, use the updates dictionary
model_obj, created = upsert(TestModel.objects, int_field=1, updates={'float_field': 3.0})
print(created)
False
print(model_obj.int_field, model_obj.float_field)
1, 3.0

# You can use updates on a newly created object that will also be used as initial values.
model_obj, created = upsert(TestModel.objects, int_field=2, updates={'float_field': 4.0})
print(created)
True
print(model_obj.int_field, model_obj.float_field)
2, 4.0

bulk_upsert¶

manager_utils.manager_utils.bulk_upsert(queryset, model_objs, unique_fields, update_fields=None, return_upserts=False, return_upserts_distinct=False, sync=False, native=False)¶

Performs a bulk update or insert on a list of model objects. Matches all objects in the queryset with the objs provided using the field values in unique_fields. If an existing object is matched, it is updated with the values from the provided objects. Objects that don’t match anything are bulk created. A user can provide a list update_fields so that any changed values on those fields will be updated. However, if update_fields is not provided, this function reduces down to performing a bulk_create on any non extant objects.

Parameters:

model_objs (list of Models) – A list of models to upsert.
unique_fields (list of str) – A list of fields that are used to determine if an object in objs matches a model from the queryset.
update_fields (list of str) – A list of fields used from the objects in objs as fields when updating existing models. If None, this function will only perform a bulk create for model_objs that do not currently exist in the database.
return_upserts_distinct (bool) – A flag specifying whether to return the upserted values as a list of distinct lists, one containing the updated models and the other containing the new models. If True, this performs an additional query to fetch any bulk created values.
return_upserts (bool) – A flag specifying whether to return the upserted values. If True, this performs an additional query to fetch any bulk created values.
sync (bool) – A flag specifying whether a sync operation should be applied to the bulk_upsert. If this is True, all values in the queryset that were not updated will be deleted such that the entire list of model objects is synced to the queryset.
native (bool) – A flag specifying whether to use postgres insert on conflict (upsert).

Signals:

Emits a post_bulk_operation when a bulk_update or a bulk_create occurs.

Examples:

# Start off with no objects in the database. Call a bulk_upsert on the TestModel, which includes
# a char_field, int_field, and float_field
bulk_upsert(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
], ['int_field'], ['char_field'])

# All objects should have been created
print(TestModel.objects.count())
3

# Now perform a bulk upsert on all the char_field values. Since the objects existed previously
# (known by the int_field uniqueness constraint), the char fields should be updated
bulk_upsert(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='0', int_field=1),
    TestModel(float_field=2.0, char_field='0', int_field=2),
    TestModel(float_field=3.0, char_field='0', int_field=3),
], ['int_field'], ['char_field'])

# No more new objects should have been created, and every char field should be 0
print(TestModel.objects.count(), TestModel.objects.filter(char_field='-1').count())
3, 3

# Do the exact same operation, but this time add an additional object that is not already
# stored. It will be created.
bulk_upsert(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
    TestModel(float_field=4.0, char_field='4', int_field=4),
], ['int_field'], ['char_field'])

# There should be one more object
print(TestModel.objects.count())
4

# Note that one can also do the upsert on a queryset. Perform the same data upsert on a
# filter for int_field=1. In this case, only one object has the ability to be updated.
# All of the other objects will be created
bulk_upsert(TestModel.objects.filter(int_field=1), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
    TestModel(float_field=4.0, char_field='4', int_field=4),
], ['int_field'], ['char_field'])

# There should be three more objects
print(TestModel.objects.count())
7

bulk_upsert2¶

manager_utils.manager_utils.bulk_upsert2(queryset, model_objs, unique_fields, update_fields=None, returning=False, ignore_duplicate_updates=True, return_untouched=False)¶

Performs a bulk update or insert on a list of model objects. Matches all objects in the queryset with the objs provided using the field values in unique_fields. If an existing object is matched, it is updated with the values from the provided objects. Objects that don’t match anything are bulk created. A user can provide a list update_fields so that any changed values on those fields will be updated. However, if update_fields is not provided, this function reduces down to performing a bulk_create on any non extant objects.

Parameters:

queryset (Model|QuerySet) – A model or a queryset that defines the collection to sync
model_objs (List[Model]) – A list of Django models to sync. All models in this list will be bulk upserted and any models not in the table (or queryset) will be deleted if sync=True.
unique_fields (List[str]) – A list of fields that define the uniqueness of the model. The model must have a unique constraint on these fields
update_fields (List[str], default=None) – A list of fields to update whenever objects already exist. If an empty list is provided, it is equivalent to doing a bulk insert on the objects that don’t exist. If None, all fields will be updated.
returning (bool|List[str]) – If True, returns all fields. If a list, only returns fields in the list. Return values are split in a tuple of created and updated models
ignore_duplicate_updates (bool, default=False) – Ignore updating a row in the upsert if all of the update fields are duplicates
return_untouched (bool, default=False) – Return values that were not touched by the upsert operation

Returns:

A list of results if returning is not False. created, updated, and untouched,: results can be obtained by accessing the created, updated, and untouched properties of the result.

Return type:

UpsertResult

Examples:

# Start off with no objects in the database. Call a bulk_upsert on the TestModel, which includes
# a char_field, int_field, and float_field
bulk_upsert2(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
], ['int_field'], ['char_field'])

# All objects should have been created
print(TestModel.objects.count())
3

# Now perform a bulk upsert on all the char_field values. Since the objects existed previously
# (known by the int_field uniqueness constraint), the char fields should be updated
bulk_upsert2(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='0', int_field=1),
    TestModel(float_field=2.0, char_field='0', int_field=2),
    TestModel(float_field=3.0, char_field='0', int_field=3),
], ['int_field'], ['char_field'])

# No more new objects should have been created, and every char field should be 0
print(TestModel.objects.count(), TestModel.objects.filter(char_field='-1').count())
3, 3

# Do the exact same operation, but this time add an additional object that is not already
# stored. It will be created.
bulk_upsert2(TestModel.objects.all(), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
    TestModel(float_field=4.0, char_field='4', int_field=4),
], ['int_field'], ['char_field'])

# There should be one more object
print(TestModel.objects.count())
4

# Note that one can also do the upsert on a queryset. Perform the same data upsert on a
# filter for int_field=1. In this case, only one object has the ability to be updated.
# All of the other objects will be created
bulk_upsert2(TestModel.objects.filter(int_field=1), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
    TestModel(float_field=4.0, char_field='4', int_field=4),
], ['int_field'], ['char_field'])

# There should be three more objects
print(TestModel.objects.count())
7

# Return creates and updates on the same set of models
created, updated = bulk_upsert2(TestModel.objects.filter(int_field=1), [
    TestModel(float_field=1.0, char_field='1', int_field=1),
    TestModel(float_field=2.0, char_field='2', int_field=2),
    TestModel(float_field=3.0, char_field='3', int_field=3),
    TestModel(float_field=4.0, char_field='4', int_field=4),
], ['int_field'], ['char_field'])

# All four objects should be updated
print(len(updated))
4

bulk_update¶

manager_utils.manager_utils.bulk_update(manager, model_objs, fields_to_update)¶

Bulk updates a list of model objects that are already saved.

Parameters:	model_objs (list of `Models`) – A list of model objects that have been updated. fields_to_update: A list of fields to be updated. Only these fields will be updated
Signals:	Emits a post_bulk_operation signal when completed.

Examples:

# Create a couple test models
model_obj1 = TestModel.objects.create(int_field=1, float_field=2.0, char_field='Hi')
model_obj2 = TestModel.objects.create(int_field=3, float_field=4.0, char_field='Hello')

# Change their fields and do a bulk update
model_obj1.int_field = 10
model_obj1.float_field = 20.0
model_obj2.int_field = 30
model_obj2.float_field = 40.0
bulk_update(TestModel.objects, [model_obj1, model_obj2], ['int_field', 'float_field'])

# Reload the models and view their changes
model_obj1 = TestModel.objects.get(id=model_obj1.id)
print(model_obj1.int_field, model_obj1.float_field)
10, 20.0

model_obj2 = TestModel.objects.get(id=model_obj2.id)
print(model_obj2.int_field, model_obj2.float_field)
10, 20.0

sync¶

manager_utils.manager_utils.sync(queryset, model_objs, unique_fields, update_fields=None, **kwargs)¶

Performs a sync operation on a queryset, making the contents of the queryset match the contents of model_objs.

This function calls bulk_upsert underneath the hood with sync=True.

Parameters:

model_objs (list of Models) – The models to sync
unique_fields – A list of fields that are used to determine if an object in objs matches a model from the queryset.
update_fields (list of str) – A list of fields used from the objects in objs as fields when updating existing models. If None, this function will only perform a bulk create for model_objs that do not currently exist in the database.
native (bool) – A flag specifying whether to use postgres insert on conflict (upsert) when performing bulk upsert.

sync2¶

manager_utils.manager_utils.sync2(queryset, model_objs, unique_fields, update_fields=None, returning=False, ignore_duplicate_updates=True)¶

Performs a sync operation on a queryset, making the contents of the queryset match the contents of model_objs.

Note: The definition of a sync requires that we return untouched rows from the upsert opertion. There is no way to turn off returning untouched rows in a sync.

Parameters:

queryset (Model|QuerySet) – A model or a queryset that defines the collection to sync
model_objs (List[Model]) – A list of Django models to sync. All models in this list will be bulk upserted and any models not in the table (or queryset) will be deleted if sync=True.
unique_fields (List[str]) – A list of fields that define the uniqueness of the model. The model must have a unique constraint on these fields
update_fields (List[str], default=None) – A list of fields to update whenever objects already exist. If an empty list is provided, it is equivalent to doing a bulk insert on the objects that don’t exist. If None, all fields will be updated.
returning (bool|List[str]) – If True, returns all fields. If a list, only returns fields in the list. Return values are split in a tuple of created, updated, and deleted models.
ignore_duplicate_updates (bool, default=False) – Ignore updating a row in the upsert if all of the update fields are duplicates

Returns:

A list of results if returning is not False. created, updated, untouched,: and deleted results can be obtained by accessing the created, updated, untouched, and deleted properties of the result.

Return type:

UpsertResult

id_dict¶

manager_utils.manager_utils.id_dict(queryset)¶

Returns a dictionary of all the objects keyed on their ID.

Return type:	dict
Returns:	A dictionary of objects from the queryset or manager that is keyed on the objects’ IDs.

Examples:

TestModel.objects.create(int_field=1)
TestModel.objects.create(int_field=2)

print(id_dict(TestModel.objects.all()))

post_bulk_operation¶

A signal that is emitted at the end of a bulk operation. The current bulk operations are Django’s update and bulk_create methods and this package’s bulk_update method. The signal provides the model that was updated.

manager_utils.post_bulk_operation = <django.dispatch.dispatcher.Signal object>¶

from manager_utils import post_bulk_operation

def signal_handler(self, *args, **kwargs):
    print kwargs['model']

post_bulk_operation.connect(signal_handler)

print(TestModel.objects.all().update(int_field=1))
<type 'TestModel'>

Method Documentation¶

single¶

get_or_none¶

upsert¶

bulk_upsert¶

bulk_upsert2¶

bulk_update¶

sync¶

sync2¶

id_dict¶

post_bulk_operation¶

Table of Contents

Previous topic

Next topic

This Page