From f74994f20828dfe15d813b3612441661fb65120f Mon Sep 17 00:00:00 2001 From: "Yakubov, Sergey" Date: Fri, 7 Feb 2025 13:46:27 -0500 Subject: [PATCH] revise advanced pydantic validation section --- .gitignore | 1 + episodes/06-Advanced-Data-Modeling.md | 204 ++++++++++++++++++++++---- 2 files changed, 177 insertions(+), 28 deletions(-) diff --git a/.gitignore b/.gitignore index 00c5e0c3..fb0f8d58 100755 --- a/.gitignore +++ b/.gitignore @@ -54,3 +54,4 @@ docs/ # translation temp files po/*~ +.idea \ No newline at end of file diff --git a/episodes/06-Advanced-Data-Modeling.md b/episodes/06-Advanced-Data-Modeling.md index 24647116..dd997e05 100755 --- a/episodes/06-Advanced-Data-Modeling.md +++ b/episodes/06-Advanced-Data-Modeling.md @@ -1,31 +1,30 @@ --- -title: "Data Validation with Pydantic" +title: "Advanced Data Validation with Pydantic" teaching: 20 exercises: 0 --- ::::::::::::::::::::::::::::::::::::::: objectives +- Represent data model using Pydantic library. - Define nested Pydantic models to represent complex data structures. -- Implement custom validation logic using Pydantic validator decorators (`@model_validator`). -- Trigger and observe validation errors for nested models. -- Apply custom validators to enforce business-specific rules. -- Explore different types of Pydantic validators (`@model_validator`, `@field_validator`, `@root_validator`) and understand their use cases. +- Implement custom validation logic for a single field. +- Implement custom validation logic for the whole model. +- Use Pydantic models in NOVA framework. :::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::: questions +- Why is data validation important? +- What is Pydantic and how it works? - How can I represent complex data structures with nested relationships using Pydantic? - How can I enforce validation rules that go beyond basic type checking using Pydantic? -- How do I define custom validation logic for Pydantic models? -- What are the different types of Pydantic validators, and when should I use each one? -- How can I trigger and observe validation errors in my Pydantic models? -- How do I use Pydantic models and validators effectively in conjunction with my viewmodel? +- How do I use Pydantic models in NOVA framework? :::::::::::::::::::::::::::::::::::::::::::::::::: -# 6. Data Validation with Pydantic: Ensuring Data Integrity +# 6. Advanced Data Validation with Pydantic: Ensuring Data Integrity In this section, we will explore Pydantic, a powerful Python library for data validation and settings management. We\'ll delve into the benefits of data validation, how Pydantic works, and best practices for using it effectively within the NOVA framework and the MVVM architecture. @@ -68,10 +67,9 @@ from pydantic import BaseModel, Field class User(BaseModel): id: int = Field(default=1, gt=0) # id must be an integer greater than 0 name: str = Field(default="someName", min_length=1) # name must be a string with at least one character - email: str = Field(default="test@test.com", regex=r"^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$") # email must be a valid email address ``` -In this example, we define a `User` model with three fields: `id`, `name`, and `email`. We use type hints to specify the data type for each field (e.g., `int`, `str`) and `Field` with validation arguments to specify additional constraints (e.g., `gt=0`, `min_length=1`, `regex=...`). +In this example, we define a `User` model with two fields: `id` and `name`. We use type hints to specify the data type for each field (e.g., `int`, `str`) and `Field` with validation arguments to specify additional constraints (e.g., `gt=0`, `min_length=1`, ...). When you create an instance of the `User` model, Pydantic automatically validates the input data: @@ -79,32 +77,147 @@ When you create an instance of the `User` model, Pydantic automatically validate from pydantic import ValidationError try: - user = User(id=0, name="", email="invalid-email") + user = User(id=0, name="") except ValidationError as e: print(e) ``` If the input data is invalid, Pydantic raises a `ValidationError` exception with detailed information about the validation errors. -## Where to Perform Validation: Model vs. ViewModel -In the MVVM architecture, the question arises: where should data validation be performed? Should it be done in the Model or the ViewModel? +## Using Pydantic to represent more complex data structures -* **Model Validation:** Validating data in the Model ensures that the underlying data is always in a valid state. This is especially important for data that is stored in a database or used by other parts of the application. However, Model validation is not always UI-specific. +When working with structured data, it\'s common to have nested objects. For example, a User model from the above example might have multiple Address entries. In Pydantic, we can achieve this by creating nested models. -* **ViewModel Validation:** Validating data in the ViewModel allows you to perform UI-specific validation, such as checking that a required field is not empty or that a value falls within a certain range. ViewModel validation is also useful for providing immediate feedback to the user about validation errors. +1. Creating the Address Model +The Address model represents a simple address with three fields: -In general, it\'s a good practice to perform both Model and ViewModel validation. Model validation ensures data integrity at the data level, while ViewModel validation provides a better user experience by providing immediate feedback and preventing invalid data from being passed to the Model. Pydantic supports both. +- street: A string with a minimum length of 3 and a maximum of 50. +- city: A string with a minimum length of 2 and a maximum of 30. +- zip_code: A string that must match a 5-digit ZIP code format. +- type: A string that must be "home" or "work". -In the context of our NOVA tutorial, here\'s how we can apply this: +```python +from typing import Literal +from pydantic import BaseModel, Field + +class Address(BaseModel): + street: str = Field(min_length=3, max_length=50) + city: str = Field(min_length=2, max_length=30) + zip_code: str = Field(pattern=r"^\d{5}$") # US ZIP code validation + type: Literal["home", "work"] = Field() +``` + +2. Using the Address Model as a Nested Field +The User model contains: + +- id: An integer that must be greater than 0 (default is 1). +- name: A required string with at least 1 character (default is "someName"). +- addresses: A list of Address models, requiring at least one address. + +```python +from typing import List + +class User(BaseModel): + id: int = Field(default=1, gt=0) + name: str = Field(default="someName", min_length=1) + addresses: List[Address] = Field(min_items=1) + +# Example input +user_data = { + "id": 1, + "name": "Alice", + "addresses": [{ + "street": "123 Main St", + "city": "New York", + "zip_code": "10001", + "type": "home" + }] +} + +user = User(**user_data) +print(user) + +``` +::::::::::::::::::::::::::::::::::::::::: callout +For easier integration with the NOVA framework, where model field information is used for displaying and validating GUI elements, we recommend avoiding overly complex nested structures. In particular, lists of lists are currently not supported. +:::::::::::::::::::::::::::::::::::::::::::::::::: + +## Implement custom validation logic for a single field. + +Sometimes, simple validation like checking the minimum length is not enough. In such cases, you can write a custom validation function for a specific field. + +For example, let\'s say we have a User model where only even IDs are allowed. We can enforce this constraint using the `@field_validator decorator`: + +```python +from pydantic import BaseModel, Field, field_validator + +class User(BaseModel): + id: int = Field(default=1, gt=0) + name: str = Field(default="someName", min_length=1) + + @field_validator("id", mode="after") + @classmethod + def is_even(cls, value: int) -> int: + if value % 2 == 1: + raise ValueError(f"{value} is not an even number") + return value + +# Example input +user_data = { + "id": 1, + "name": "Alice", +} + +user = User(**user_data) +print(user) + +``` + +This code will raise a ValueError because the provided id (1) is not an even number. -* **NDIP Interactions (Model):** When using `nova-galaxy` to interact with NDIP, validate the data being sent to NDIP in the Model to ensure it conforms to the NDIP API requirements. -* **UI Input (ViewModel):** When the user enters data in the UI, validate the data in the ViewModel to provide immediate feedback to the user. +::::::::::::::::::::::::::::::::::::::::: callout +Note that we used the mode="**after**" option for the validator. This ensures that our custom validation runs after Pydantic\'s internal validation (in our example example, checking that the id is an integer and greater than 0). Alternatively, you can use mode="**before**", where custom validation occurs before the internal validation. Validators in the after mode are generally more type-safe, making them easier to implement. +:::::::::::::::::::::::::::::::::::::::::::::::::: + + +## Implement custom validation logic for the whole model. + +In some cases, you may need to validate the entire model, not just individual fields. This can be done by writing a custom validation function for the whole model using the `@model_validator` decorator. + +For example, let\'s say we have a User model where the name and id must meet specific conditions together. For instance, we only allow users with even IDs to have names that start with a capital letter. We can enforce this logic using a @model_validator: + +```python +from pydantic import BaseModel, Field, model_validator +from typing_extensions import Self + +class User(BaseModel): + id: int = Field(default=1, gt=0) + name: str = Field(default="someName", min_length=1) + + @model_validator(mode='after') + def check_name_for_even_id(self) -> Self: + if self.id % 2 == 0 and not self.name[0].isupper(): + raise ValueError(f"Name must start with a capital letter when the ID is even.") + + return self + +# Example input +user_data = { + "id": 2, + "name": "alice", # Name starts with lowercase, should raise an error +} + +user = User(**user_data) +print(user) +``` + +This code will raise a ValueError because the name ("alice") does not start with a capital letter, while the id is even. -## Pydantic and `nova-trame` Input Validation +## Using Pydantic models in NOVA framework -One of the great features of `nova-trame` is that it leverages the validation attributes of Pydantic models to automatically create validation routines for Vuetify UI elements. Let\'s walk through what that looks like in code. +One of the great features of the NOVA Framework is that it allows to leverage Pydantic models to automatically validation UI elements. Let\'s walk through what that looks like in code. First, let\'s assume you have the following model: @@ -112,10 +225,23 @@ First, let\'s assume you have the following model: from pydantic import BaseModel, Field class SettingsModel(BaseModel): - port: int = Field(default=8080, gt=0, lt=65536, title="Port Number", description="The port to listen on.") + port: int = Field(default=8080, gt=0, lt=65536, title="Port Number", description="The port to listen on.", examples=["12345"]) ``` -Then in your view, you create the following InputField: +Then in your viewmodel, you create binding for this model: + +```python +from nova.mvvm.interface import BindingInterface +class ConfigViewModel: + def __init__(self, binding: BindingInterface): + self.settings = SettingsModel() + self.settings_bind = binding.new_bind(self.settings) + + def update_view(self): + self.settings_bind.update_view(self.settings) +``` + +And in your view, you create the following InputField: ```python from nova.trame.view.components import InputField @@ -123,8 +249,30 @@ from nova.trame.view.components import InputField InputField(v_model="settings.port") ``` -Notice how you don\'t need to pass any attributes to `InputField` other than `v_model`. The `InputField` automatically retrieves the `title` and attempts to retrieve other information. +Notice how you don\'t need to pass any attributes to `InputField` other than `v_model`. The `InputField` automatically retrieves the `title`, `description` and `examples` and uses these values for label, hint and empty value. + +The InputField also performs automatic validation for this field. If you enter an invalid port number into the InputField, the InputField will change state to invalid and the label will turn red. + +In that fashion, the `InputField` seamlessly pulls information from your code\'s data model and displays errors to the user. -The InputField performs automatic validation for this field. If you enter an invalid port number into the InputField, the InputField will change state to invalid and the label will turn red. +### Using callbacks in ViewModel to react to validation errors + +Sometimes, you may want to respond to UI validation errors beyond just marking a field as invalid (which happens automatically). In such cases, you can add a callback to the `new_bind` function: + +```python +from typing import Any, Dict +from nova.mvvm.interface import BindingInterface +class ConfigViewModel: + def __init__(self, binding: BindingInterface): + self.settings = SettingsModel() + self.settings_bind = binding.new_bind(self.settings, callback_after_update=self.process_settings_change) + + def process_settings_change(self, results: Dict[str, Any]) -> None: + if results["error"]: + print(f"error in fields {results['errored']}, model not changed") + else: + print(f"model fields updated: {results['updated']}") + +``` -In that fashion, the `InputField` seamlessly pulls information from your code\'s data model and displays errors to the user. \ No newline at end of file +The function will receive a dictionary containing lists of updated or invalid fields. Note that if a validation error occurs, the model will not be updated, leading to a discrepancy between the values displayed in the UI and those in the model. \ No newline at end of file -- GitLab