tricoteuses-assemblee

Tricoteuses-Assemblee

Retrieve, clean up & handle French Assemblée nationale's open data

Retrieval of open data (in JSON format) from Assemblée nationale's website

npx babel-node --extensions ".ts" --max-old-space-size=5120 -- src/scripts/retrieve_open_data.ts ../assemblee-data/

Splitting of open data into small files

npx babel-node --extensions ".ts" --max-old-space-size=2048 src/scripts/split_data.ts ../assemblee-data/

Note: These split files are also available in Tricoteuses / Open data Assemblée nationale repositories. They are updated every day.

Validation & cleaning of JSON data

Validation & cleaning of small split files

npx babel-node --extensions ".ts" -- src/scripts/clean_split_data.ts  ../assemblee-data/

Note: These split & cleaned files are also available in Tricoteuses / Open data Assemblée nationale repositories with the _nettoye suffix. They are updated every day.

Validation & cleaning of big non-split files

npx babel-node --extensions ".ts" --max-old-space-size=8192 -- src/scripts/clean_data.ts ../assemblee-data/

Note: The big non-split open data files should not be used. Use small split files instead.

Retrieval of députés' pictures from Assemblée nationale's website

npx babel-node --extensions ".ts" -- src/scripts/retrieve_deputes_photos.ts --fetch ../assemblee-data/

Retrieval of députés' non open data informations from Assemblée nationale's website

npx babel-node --extensions ".ts" -- src/scripts/retrieve_deputes_infos.ts --fetch --parse ../assemblee-data/

Retrieval of documents from Assemblée nationale's website

npx babel-node --extensions ".ts" -- src/scripts/retrieve_documents.ts --fetch --parse ../assemblee-data/

Test loading everything in memory

Test loading small split files

npx babel-node --extensions ".ts" --max-old-space-size=2048 -- src/scripts/test_load_all.ts ../assemblee-data/

Test loading big non-split files

npx babel-node --extensions ".ts" --max-old-space-size=2048 -- src/scripts/test_load_all_big_files.ts ../assemblee-data/

Note: The big non-split open data files should not be used. Use small split files instead.

Initial generation of TypeScript files from JSON data.

npx quicktype --acronym-style=camel -o src/raw_types/acteurs_et_organes.ts ../assemblee-data/AMO*_{XIV,XV,historique}.json

Edit src/raw_types/acteurs_et_organes.ts to add the following static methods to class Convert:

    public static toActeur(json: string): Acteur {
        return cast(JSON.parse(json), r("Acteur"));
    }

    public static acteurToJson(value: Acteur): string {
        return JSON.stringify(uncast(value, r("Acteur")), null, 2);
    }

    public static toOrgane(json: string): Organe {
        return cast(JSON.parse(json), r("Organe"));
    }

    public static organeToJson(value: Organe): string {
        return JSON.stringify(uncast(value, r("Organe")), null, 2);
    }

npx quicktype --acronym-style=camel -o src/raw_types/agendas.ts ../assemblee-data/Agenda_{XIV,XV}.json

Edit src/raw_types/agendas.ts to add the following static methods to class Convert:

    public static toReunion(json: string): Reunion {
        return cast(JSON.parse(json), r("Reunion"));
    }

    public static reunionToJson(value: Reunion): string {
        return JSON.stringify(uncast(value, r("Reunion")), null, 2);
    }

npx quicktype --acronym-style=camel -o src/raw_types/amendements.ts ../assemblee-data/Amendements_XV.json

Edit src/raw_types/amendements.ts to add the following static methods to class Convert:

    public static toTexteleg(json: string): Texteleg {
        return cast(JSON.parse(json), r("Texteleg"));
    }

    public static textelegToJson(value: Texteleg): string {
        return JSON.stringify(uncast(value, r("Texteleg")), null, 2);
    }

npx quicktype --acronym-style=camel -o src/raw_types/dossiers_legislatifs.ts ../assemblee-data/Dossiers_Legislatifs_{XIV,XV}.json

Edit src/raw_types/dossiers_legislatifs.ts to add the following static methods to class Convert:

    public static toDocument(json: string): Document {
        return cast(JSON.parse(json), r("Document"));
    }

    public static documentToJson(value: Document): string {
        return JSON.stringify(uncast(value, r("Document")), null, 2);
    }

    public static toDossierParlementaire(json: string): DossierParlementaire {
        return cast(JSON.parse(json), r("DossierParlementaire"));
    }

    public static dossierParlementaireToJson(value: DossierParlementaire): string {
        return JSON.stringify(uncast(value, r("DossierParlementaire")), null, 2);
    }

npx quicktype --acronym-style=camel -o src/raw_types/scrutins.ts ../assemblee-data/Scrutins_{XIV,XV}.json

Edit src/raw_types/scrutins.ts to add the following static methods to class Convert:

    public static toScrutin(json: string): Scrutin {
        return cast(JSON.parse(json), r("Scrutin"));
    }

    public static scrutinToJson(value: Scrutin): string {
        return JSON.stringify(uncast(value, r("Scrutin")), null, 2);
    }