10.04.2023 - Page 1
DépartementOfficeApp::AutoCRUD
An application for searching and manipulating data in relational
databases
YAPC::EU::2014
[email protected] de Genève, Pouvoir Judiciaire
DépartementOffice
Agenda
• Introduction• Overview (screenshots)• Getting started• Architecture• Extensibility• Conclusion
10.04.2023 - Page 1
DépartementOffice
Introduction
Business need : tool for the support team
RegularUsers
database
RegularApp
DatabaseAdmins
AdminTools
SupportTeam
? ? ?
Data access for support team
Needs• Browse DB schema• CRUD on all data• Navigate
– Follow relationships– Optimize for most
common tables and colums
Solutions• Program an admin
GUI– too much work !
• Give them DBA tools– too low-level– no customization
Need a CRUD app !
CRUD landscape
Admin toolsOracle SQLDeveloperSQLite SqlitemanPostgres pgadminphpMyAdmin...
Readyto use
Needsprogramming
Fixedfeatures
Customizable
Scaffolding toolsRuby on RailsCatalystCatalystX::CRUD...
AutoCRUD appsCatalyst::Plugin::AutoCRUD (2011)App::AutoCRUD (Jan 2014)RapidApp (Feb 2014) [ WebAPI::DBIC ] (Jul 2014)Interchange TableEditor (2013??)
App::AutoCRUD main features
• Powerful search syntax• Distinctive URL for every resource
– no session, all information in URL– easy admin links from the "real App"
• Joins as hyperlinks• Bulk update / delete operations• Customizable table order / column order• Navigation through tree navigator
10.04.2023 - Page 1
DépartementOffice
Quick overview
To run the demo :cd examplesperl download_db.plplackup
Homepage
Specified in config file
http://chinookdatabase.codeplex.com/https://code.google.com/p/sakila-sample-database-ports/
Demo: free sample
databases
Datasource entry point
Auto-generateddatamodel
Grouping, order & comments
from config file
Table description
Grouping, order & comments
from config file
Hyperlinks to foreign tables
from DB schema
Search formMultiple values
(OR)
LIKE
Multiple views
Choose columns
BETWEEN
Results from search
Hyperlinks to foreign records or lists
Only show non-null columns
Also navigate through LEFT/RIGHT arrow keys
Excel output
YAML output
Bulk update (selective)
2) Call the update form
1) Check records to be modified
Update form
Keys of selected records
Bulk update (from WHERE criteria)
Call the update form
Update form
Criteria for the update
Fields to update
± RESTful URLs
• /Chinook/table/Track/search?GenreId=1,3&Name=Ab*– GET display pre-filled search form– POST redirect to list
• /Chinook/table/Track/list?GenreId=1,3&Name=Ab*– search results
• /Chinook/table/Track/id/399– single record
• /Chinook/table/Track/delete?where_pk.TrackId=399– GET display pre-filled delete form– POST database operationNote : browser-oriented, not API-oriented
No use of HTTP methods PUT and DELETE
10.04.2023 - Page 1
DépartementOffice
Getting Started
YAML Config : connection parameters
app: name: Test AutoCRUD
datasources : Source1 : dbh: connect: - dbi:SQLite:dbname=some_file - "" # user - "" # password - RaiseError : 1 sqlite_unicode: 1
Config : specifying table grouping / ordering
tablegroups : - name: Music descr: Tables describing music content node: open tables : - Artist - Album - Track
- name: Playlist ...
Config : specifying col grouping / ordering
Employee: colgroups: - name: keys columns: - name: EmployeeId descr: primary key - name: ReportsTo descr: foreign key to the manager - name: Employee details columns: - name: Title - name: FirstName - name: LastName
"crud.psgi" startup file
use App::AutoCRUD;use YAML qw/LoadFile/;
my $config = LoadFile "/path/to/config.yaml";my $crud = App::AutoCRUD->new(config => $config);
my $app = $crud->to_app;
Start the application
• From the command lineplackup crud.psgi
• Within Apache<Location /crud>
SetHandler perl-scriptPerlResponseHandler Plack::Handler::Apache2 PerlSetVar psgi_app /path/to/crud.psgi
</Location>
• Other possibilities : see L<Plack>
10.04.2023 - Page 1
DépartementOffice
Architecture : external modules
External modules : Global view
TemplateToolkit
Alien::GvaScript
Prototype.js
Excel::Writer:
:XLSX
View
DBIx::DataModel
DBI
SQL::Abstract::More
Model
Plack
CGI::Expand
SQL::Abstract:
:FromQuery
Controller
Infrastructure
MooseData::Domai
nYAML
Infrastructure modules
• Moose for OO– Modern Perl OO framework
• YAML for config data– Human-readable– Support for directed acyclic graph (DAG) reusable subtrees– But can be replaced by JSON, XML, Config::General or Perl
source code
• Data::Domain for data validation
Controller modules
• Plack for HTTP support– Abstraction over various engines– Many middleware options
• CGI::Expand for transforming form inputs – <input name="foo.1.bar" value=1234>– { foo => [ {...}, { bar => 1234 } ] }
• SQL::Abstract::FromQuery for parsing user queries– HTML form SQL::Abstract::More SQL– Parses booleans, < > !=, LIKE, BETWEEN, etc.
Model modules
• DBI for database independance– Perl standard for databases
• DBIx::DataModel for object-relational mapping – Why not DBIC ? See
http://www.slideshare.net/ldami/dbixclass-vs-dbixdatamodel
• SQL::Abstract::More for SQL generation– Emit SQL from Perl datastructures
View modules
• Template Toolkit for HTML generation
• Alien::GvaScript for widgets – Tree::Navigator
• Prototype.js for Javascript abstraction– Navigator abstraction, OO programming, etc.
• Excel::Writer::XLSX for generating Excel worksheets
10.04.2023 - Page 1
DépartementOffice
Architecture : internal structure
Internal structure : global view
AutoCRUD
AutoCRUD:
:View
AutoCRUD:
:Controller
AutoCRUD:
:Context
AutoCRUD:
:DataSource
Plack::Component
Moose::Object
Controller::Table
Controller::Schema
View::XLSX
View::TT
...
App::AutoCRUD
• Entry point• Autodiscovers component subclasses• isa Plack::Component• Has
– Name– Config– Datasources
App::AutoCRUD::DataSource
• Encapsulates DBIx::DataModel schema– Autogenerate Perl code if not already present
• Gathers and merges metadata– From the DB schema
• Tables, columns, foreign keys– From config
• Table groups, column groups, ordering, display parameters
App::AutoCRUD::Context
• Gathers contextual information for the current request– Plack::Request object– Datasource object– View to apply– Parameters for the view (for example TT template name)
Controller::Schema
• Lists table groups & tables• Publishes the DBIx::DataModel schema
Controller::Table
• Implements CRUD operations• Methods
– Descr– List– Id– Search– Update– Delete– Insert
View::*
• Render results– TT– XLSX– YAML– JSON– ...
10.04.2023 - Page 1
DépartementOffice
Extensibility
Regular subclassing
• Extend the apppackage My::Private::AutoCRUD;use Moose;extends 'App::AutoCRUD';
• Subclass and redefine existing methods– Automatic subclass discovery– Good candidates :
• DataSource::prepare_for_request • DataSource::_query_parser
Adding URLs
• New controllers– Crud/<db_name>/<controller_name>/....
• New methods in existing controllers– Crud/<db_name>/table/<table_name>/<method>/...
Adding Views
• Create View/XYZ.pm with "render" method– Serves /path/to/resource.xyz?p1=v1&p2=v2&...
Relooking
• Override CSS – My/Private/AutoCRUD/share/static/css/styles.css
• Override TT templates – My/Private/AutoCRUD/share/templates/src/table/list.tt
10.04.2023 - Page 1
DépartementOffice
Conclusion
Status
• Still young, lots of TODO• Already in production at Geneva Justice• Has some original features not seen other CRUD
apps• Could be useful for many people
– Also outside the Perl community • Perl marketing needs general-purpose apps
– Needs stabilization & doc– Needs marketing– You can help !