git.net

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Refactor models.py


Hi Fokko,

I commented on some of the issues -we could possibly just delete User and KnownEvent*

> My suggestion is to create a new package, called models, which will contain all the orm classes

And do what with the current airflow.models?

Do you have an idea of module names to move things to? Are you thinking we have airflow.models.connection module containing just a Connection class for example?

-ash

> On 6 Dec 2018, at 11:35, Driesprong, Fokko <fokko@xxxxxxxxxxxxxx> wrote:
> 
> Hi All,
> 
> I think it is time to refactor the infamous models.py. This file is far too
> big, and it only keeps growing. My suggestion is to create a new package,
> called models, which will contain all the orm classes (the ones
> with __tablename__ in the class). And for example the BaseOperator to the
> operator packages. I've created a lot of tickets to move the classes one by
> one out of models.py. The reason to do this one by one is to relieve the
> pain of fixing the circular dependencies.
> 
> Refactor: Move DagBag out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3456
> 
> Refactor: Move User out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3457
> 
> Refactor: Move Connection out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3458
> 
> Refactor: Move DagPickle out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3459
> 
> Refactor: Move TaskInstance out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3460
> 
> Refactor: Move TaskFail out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3461
> 
> Refactor: Move TaskReschedule out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3462
> 
> Refactor: Move Log out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3463
> 
> Refactor: Move SkipMixin out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3464
> 
> Refactor: Move BaseOperator out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3465
> 
> Refactor: Move DAG out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3466
> 
> Refactor: Move Chart out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3467
> 
> Refactor: Move KnownEventType out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3468
> 
> Refactor: Move KnownEvent out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3469
> 
> Refactor: Move Variable out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3470
> 
> Refactor: Move XCom out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3471
> 
> Refactor: Move DagStat out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3472
> 
> Refactor: Move DagRun out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3473
> 
> Refactor: Move SlaMiss out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3474
> 
> Refactor: Move ImportError out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3475
> 
> Refactor: Move KubeResourceVersion out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3476
> 
> Refactor: Move KubeWorkerIdentifier out of models.py
> https://issues.apache.org/jira/browse/AIRFLOW-3477
> 
> Some classes are really simple, and would also be a nice opportunity for
> newcomers to start contributing to Airflow :-)
> 
> Cheers, Fokko