Replies: 7 comments 17 replies
-
Iterating relationsOrmar uses same method to iterate nested models and avoid going into infinite recursion loop. The same one is use in Meaning in schema like this: A # <= this A wont be saved/loaded as root node is A, but if X have a fk column to A it will be saved
/
B - C - D - X - Z
/
A - C - U - X - O - K
\
U # <= this U won't be saved/loaded as U was already between C and X in this branch, but fk value in O will be savedSaving nested modelsBy default only the main models are saved, but you can save deeply related models in at least two ways -> depending on your data structure. One way is That's why you have signals (pre/post save/update/delete), where you get sender class (model), current instance etc. So for example you can define signal receiver function so that D is always saved in pre_save/pre_update hook each time the main model is saved/updated and rest is managed by Fetching nested modelsIf you query the db with Oracle and Couchbase supportUnfortunately |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @collerek ,appreciated your reply . I'm looking for solution for transforming data coming from relational DB to non-relation DB , so for my previous example - it does work (with some annoying WA and some limitation BUT -it is complex for the user to define all those relation, and kind of tricky to maintain . |
Beta Was this translation helpful? Give feedback.
-
|
I feel some unclearness for the requirement ormar has for one2many, one2one relation . Maybe i'm not looking in the right place, i.e: I tried to find an example of saved_related /select_all/ on documentation , I tried building quite nested objects but got messed up got confuse with the way ormar creates the object relation, Code: referring to example of nested object, following ur explanation for limitation of "repeatable " objects. customer can have one address (one2one), Json can look like: Do you see an issue describing this module with ormar? |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Department exampleThe There is no Below you have a working example for your department and course example (wrapped in pytest test). import json
from typing import List, Optional
import databases
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
metadata = sqlalchemy.MetaData()
database = databases.Database(DATABASE_URL, force_rollback=True)
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class Department(ormar.Model):
class Meta:
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
department_name: str = ormar.String(max_length=100)
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
course_name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean()
department: Optional[Department] = ormar.ForeignKey(Department)
# create db and tables
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
@app.post("/DepartmentWithCourses/", response_model=Department)
async def create_department(department: Department):
# there is no save all - you need to split into save and save_related
await department.save()
await department.save_related(follow=True, save_all=True)
return department
@app.get("/DepartmentsAll/", response_model=List[Department])
async def get_Courses():
# if you don't provide default name it related model name + s so courses not course
departmentall = await Department.objects.select_related("courses").all()
return departmentall
def test_saving_related_in_fastapi():
client = TestClient(app)
with client as client:
payload = {
"department_name": "Ormar",
"courses": [
{
"course_name": "basic1",
"completed": True
}, {
"course_name": "basic2",
"completed": True
}]
}
response = client.post("/DepartmentWithCourses/", data=json.dumps(payload))
department = Department(**response.json())
assert department.id is not None
assert len(department.courses) == 2
assert department.department_name == 'Ormar'
assert department.courses[0].course_name == "basic1"
assert department.courses[0].completed
assert department.courses[1].course_name == "basic2"
assert department.courses[1].completed
response = client.get("/DepartmentsAll/")
departments = [Department(**x) for x in response.json()]
assert departments[0].id is not None
assert len(departments[0].courses) == 2
assert departments[0].department_name == 'Ormar'
assert departments[0].courses[0].course_name == "basic1"
assert departments[0].courses[0].completed
assert departments[0].courses[1].course_name == "basic2"
assert departments[0].courses[1].completedNested relation exampleThe repeatable Model in schema can be a problem if you have a loop, that's why Your schema looks like follow: Now when you split it into relation "chains" you have 3:
As you can see no model repeats itself in any branch/relation chain. Problem would be if you have for example something like: Now, everything would work as supposed to if you split the db calls in pieces, but with schema like above if you would like to issue one giant json up to Customer Responsible it's not possible, cause you have a list of customers assigned to a Region, and now each of those customers should have Address, Subscriber Type(1), Credit Plan ... etc. and you just go deeper and deeper into infinite loop. Again splitted in chains you have 3 relation branches:
In chain 2 and 3 Customer Model repeats so the chain will break at Region. Of course any shorten chain is fine so in example:
can be one json. Hope that clarifies the issue. |
Beta Was this translation helpful? Give feedback.
-
|
@collerek - thanks much for your detailed reply , ok now I understand why it required first to have the Parent object save before preforming save_all, my Q - if there were more child below courses , is that means we need do save each dependencies as well? In case yes and there's no other way to simplify the process ... as far as i search it seem this functionality required by many, @scd75 provide some alternate declarative_base() (sqlalchamy orm) : I tried to implement it but seems it provide partials solution - so maybe It been more than a month that I search for " ORM package for purpose i explain, I'd like to share with you some suggestion if i may , I wonder if it can combine to single solution or maybe implement in similar fashion , pyantic to sqlalchemy I think it would be really awesome if the tool will do the creation of orman model automatically again, thanks much for your assistant and explanation . |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
as not as regular Base model , ormar use both metadata and database , I'm trying to implement ormar_model in fastapi in same fashion ., main.py : routers: few ormar models which both metadata , database are not decaled here , Thanks |
Beta Was this translation helpful? Give feedback.
-
|
You don't need inheritance but you cannot declare them in main as you will create circular import: You should have one file with db that contain metadata and database. You can also create So your project could look like this: # database.py: - note there is no import from your app so it won't create a circle
import databases
import ormar
import sqlalchemy
DATABASE_URL = "sqlite://"
database = databases.Database(DATABASE_URL)
metadata = sqlalchemy.MetaData()
class BaseMeta(ormar.ModelMeta):
database = database
metadata = metadataNow in your main you need only db so: # main.py
... # other imports
from app.database import database
... # fastapi app creation etc
app.state.database = database
... # events with shutdown etc. and rest of fileand in each of models files you need only BaseMeta now: # app.module_1.models.py
import ormar
from app.database import BaseMeta
class Example(ormar.Model):
class Meta(BaseMeta):
pass
id: int = ormar.Integer(primary_key=True)
department_name: str = ormar.String(max_length=100) |
Beta Was this translation helpful? Give feedback.
-
Trying to use the automap/auto ormar class generation from posgress as following: from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
from sqlalchemy import create_engine
import ormar
Base = automap_base()
# engine, suppose it has tables setup
engine = create_engine("postgresql://admin:[email protected]:5432/customer_trg")
# reflect the tables
Base.prepare(engine, reflect=True)
# translate whole db
ormar_models = [sqlalchemy_to_ormar(sqlalchemymodel, database=database, metadata=metadata) for sqlalchemymodel in Base.classes]Getting: Data\Local\Programs\Python\Python38-32\lib\site-packages\sqlalchemy\util\deprecations.py", line 139, in warned
return fn(*args, **kwargs)
File "C:\Users\AviKarin.AVI.000\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sqlalchemy\orm\mapper.py", line 721, in __init__
self._configure_properties()
File "C:\Users\AviKarin.AVI.000\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sqlalchemy\orm\mapper.py", line 1518, in _configure_properties
self._configure_property(
File "C:\Users\AviKarin.AVI.000\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sqlalchemy\orm\mapper.py", line 1757, in _configure_property
prop = self._property_from_column(key, prop)
File "C:\Users\AviKarin.AVI.000\AppData\Local\Programs\Python\Python38-32\lib\site-packages\sqlalchemy\orm\mapper.py", line 1938, in _property_from_column
raise sa_exc.ArgumentError(
sqlalchemy.exc.ArgumentError: WARNING: when configuring property 'customer' on mapped class subscriber->subscriber, column 'customer' conflicts with property '<RelationshipProperty at 0x48b32e8; customer>'. To resolve this, map the column to the class under a different name in the 'properties' dictionary. Or, to remove all awareness of the column entirely (including its availability as a foreign key), use the 'include_properties' or 'exclude_properties' mapper arguments to control specifically which table columns get mapped.customer table: subscriber |
Beta Was this translation helpful? Give feedback.
-
|
@collerek , Trying to post getting error for "DatabaseBackend is not running": Main: Router: |
Beta Was this translation helpful? Give feedback.
-
|
You are testing it live or with
|
Beta Was this translation helpful? Give feedback.
-
models are being inherited from MetaBase as you suggest, I double check over and over and only after i boot postgres docker I'm getting the response with the id fields in each level of the json , but it didn't remove the "id". Have some "crazy" question (it's 01:30 AM here and im fully into ormar:)) : lets say i have some 5 tables which module as ormar modules. Do u think it optional with ormar to do kind of conditional commit , |
Beta Was this translation helpful? Give feedback.
-
|
Ok, so database need to be running and available if you connect to db in docker it need to run. In queries you cannot exclude pk, in the same docs you have: If you want to exclude pk you can do so in What you call conditional insert is a database transaction (check #146 ) Transactions are supported through There is really no way to make it work through several endpoints - they are async, so can run in any order, know nothing about each other and some api calls might never come to your application (internet error or whatever), so other calls have no way to determine if other calls were successful, how long would you have to wait to make sure it's not delayed etc. Apart from that it would be a bad practice, if you have several enpoints it means each of them is a separate "transaction", otherwise why bother and not merge the api endpoints into one? BTW. I already modified |
Beta Was this translation helpful? Give feedback.
-
|
for models: While using the input Json: Confirm on DB: But when Im fetching the data for Deparment (with select_related), By querying the DB: Q: 1.Is there way to get the Json for the full Department with all tree exactly as i used for inputs? I looked on the link regarding using the exclude but i didn't manage to suppress the id, where the response coming as list of Department so it failed , I think It would be more convince and understandable if ormar would provide a Class or build function While applying filter on child working with select_related, sometime like: thanks! |
Beta Was this translation helpful? Give feedback.
-
|
If you want to exclude not only value but whole field you have two options:
Yes by default you only join what you You query from main model (in example here is a sample that returns exactly the same data as used to create the department. import json
from typing import List, Optional
import databases
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
metadata = sqlalchemy.MetaData()
database = databases.Database(DATABASE_URL, force_rollback=True)
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class Department(ormar.Model):
class Meta:
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
department_name: str = ormar.String(max_length=100)
class Course(ormar.Model):
class Meta:
database = database
metadata = metadata
id: int = ormar.Integer(primary_key=True)
course_name: str = ormar.String(max_length=100)
completed: bool = ormar.Boolean()
department: Optional[Department] = ormar.ForeignKey(Department)
# create db and tables
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
to_exclude = {"id", "courses__id"}
@app.post("/departments/", response_model=Department)
async def create_department(department: Department):
# note I use the latest not released version you still need await department.save() here before save_related()
await department.save_related(follow=True, save_all=True)
return department
# note lack of response_model
@app.get("/departments/{department_name}")
async def get_department(department_name: str):
# note names of main model Department are available without `__`
department = await Department.objects.select_all().get(
department_name=department_name)
# note I use to_exclude defined above, if you had more nested models you need to add more i.e. courses__students__id
return department.dict(exclude=to_exclude)
def test_saving_related_in_fastapi():
client = TestClient(app)
with client as client:
payload = {
"department_name": "Ormar",
"courses": [
{"course_name": "basic1", "completed": True},
{"course_name": "basic2", "completed": True},
],
}
response = client.post("/departments/", data=json.dumps(payload))
department = Department(**response.json())
assert department.id is not None
assert len(department.courses) == 2
assert department.department_name == "Ormar"
assert department.courses[0].course_name == "basic1"
assert department.courses[0].completed
assert department.courses[1].course_name == "basic2"
assert department.courses[1].completed
response = client.get("/departments/Ormar")
assert response.json() == payload |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @collerek , seems select_all() would be better way for my purpose as at current im interested to those Pk/FK being added/generated automatically , |
Beta Was this translation helpful? Give feedback.
-
|
You don't need Department__id, you just had to remove the There is no way to auto exclude fields for pk/fk, a simple helper class (that relies on internal If your pk field name is always def auto_exclude_id_field(to_exclude: Any) -> Union[Dict, Set]:
if isinstance(to_exclude, dict):
for key in to_exclude.keys():
to_exclude[key] = auto_exclude_id_field(to_exclude[key])
to_exclude["id"] = Ellipsis
return to_exclude
else:
return {"id"}
def generate_exclude_for_ids(model: Type[ormar.Model]) -> Dict:
to_exclude_base = translate_list_to_dict(model._iterate_related_models())
return auto_exclude_id_field(to_exclude=to_exclude_base)
to_exclude_auto = generate_exclude_for_ids(model=Department)
# this will generate for you a dictionary cause you can also use nested dict instead of set of strings
# in example to_exclude_auto = {'courses': {'id'}, 'id': Ellipsis} which is equivalent of {"id", "courses__id"}With the helper you can pass main model (from which you query and it should exclude id in all nested models. |
Beta Was this translation helpful? Give feedback.
-
I think this functionality should be build in ormar :) I guess im not implement this right .... getting: raise RelationshipInstanceError(
ormar.exceptions.RelationshipInstanceError: Relationship error - ForeignKey Address is of type <class 'int'> while <class 'set'> passed as a parameter. |
Beta Was this translation helpful? Give feedback.
-
|
I can add two flags as for your code you should pass it to exclude in # here is the missing import
from ormar.queryset.utils import translate_list_to_dict
def auto_exclude_id_field(to_exclude: Any) -> Union[Dict, Set]:
if isinstance(to_exclude, dict):
for key in to_exclude.keys():
to_exclude[key] = auto_exclude_id_field(to_exclude[key])
to_exclude["id"] = Ellipsis
return to_exclude
else:
return {"id"}
# use the translate_list_to_dict provided by ormar as it's not that simple as you wrote it ( you didn't include nested dicts etc.)
def generate_exclude_for_ids(model: Type[ormar.Model]) -> Dict:
to_exclude_base = translate_list_to_dict(model._iterate_related_models())
return auto_exclude_id_field(to_exclude=to_exclude_base)
# btw if you do not modify the object it should not be a put request (rather get)
# if you exclude whole fields no response_model
@ormar_router.put("/CopyCustomer/{customer_no}")
async def copy_customer(customer_no: int):
customer = await Customer.objects.select_all().get(customer_no=customer_no)
# return dict and pass generate_exclude... to it, note that you pass the Model not instance
return customer.dict(exclude=generate_exclude_for_ids(model=Customer)) |
Beta Was this translation helpful? Give feedback.
-
|
@collerek @ormar_router.get("/CustomerList/",tags=["customer_on_db"])
async def get_Customers():
customers = await Customer.objects.select_all(follow=True).all()
return list(customers) |
Beta Was this translation helpful? Give feedback.
-
|
customer is already a list, so if you want to exclude in each element you need to iterate and exclude in each. @ormar_router.get("/CustomerList/",tags=["customer_on_db"])
async def get_Customers():
customers = await Customer.objects.select_all(follow=True).all()
return [x.dict(exclude=your_exclude) for x in customers] |
Beta Was this translation helpful? Give feedback.
-
|
Another question related to nested object , Routes: When open the openApi endpoint in example value While with pydantic model it is showing related objects . |
Beta Was this translation helpful? Give feedback.
-
|
That's because Since PkOnly models are kind of technical internal thing can you open a new enhancement issue (so not here in discussion but in issues) in ormar, like: "Provide meaningful example for nested models in openapi schema". I will look what I can do about this. |
Beta Was this translation helpful? Give feedback.
-
|
BTW. Can you also post here your questions in separate discussions from email (bulk creation of whole tree, sub classing ormar models with pydantic models (and any other if I missed something). I'm getting lost what is where and you ask questions that might be useful to others so I would prefer to keep them here for future reference :) |
Beta Was this translation helpful? Give feedback.
-
Interested replacing sqlalachamy and ormar looks promising:)
need your kind help understanding if ormar is my best alternative :
1.Most important - support a complex/nested objects ?
case yes , is there any limitation for nested level depth or types.
The complexity of data i have can be really challenge ,
So for post request with a nested object as input,
can ormar read the nested json i.e ,and populated to the relevant tables (by predefine models)?
i.e :
Beta Was this translation helpful? Give feedback.
All reactions