Statistics 2 go

(simplePXsql, PX2go and API)

PC-Axis reference group, Tirana 2013 Lars Pedersen

How many persons are living in Greenland ?

Look to:

http://bank.stat.gl/beest1

Question:

Answer by a pc:

Answer by an iPad :

Answer by an iPhone:

• Motorola Storno• Sony Ericsson• Nokia

• and px-web 2008

Think off:

How many persons are living in Greenland ?

http://bank.stat.gl/beest1

http://bank.stat.gl/m.asp?m=beest1

Question:

Answer by a pc:

Answer by an iPad:

Overview

• a little history• from px-files to a database• what is that good for?• px2go• api’s

• cliffhanger

a little history :

the family concept

• We need to stay connected• Meet, party and quarrel

• a road-map

the family concept

the family concept

PX-files

DBNordic

PX-webPC-Axis

PC-Axis/sqlDenmarkNorwayIreland…

Createprocedures

Createprocedures

Createprocedures

Createprocedures

?

Database specific settings:

[System]sql_database=statbank2013sql_baseowner=..sql_datestamp=Y-D-M h:m:00sql_endline=sql_bulksave=1

from px-files to a database

All can be done batch by a few lines in

PX-JOB

from px-files to a database

@echo off: Batch command line:: PX-Job.exe {job} [in] {out} {err} {def} {log} {-options}:PX-Job job=sql in=H:\database\test\ out=H:\database\test\ log=H:\database\test\pxjob1.txt err=H:\database\test\ -s -o1 -e1PX-Job job=sql in=H:\database\test\ out=H:\database\test\ log=H:\database\test\pxjob1a.txt err=H:\database\test\ -s -o3 -e1

input

approximately 650 px filesnormal tree structure

output with switch -s -o1 -e1

INSERT INTO statbank2013..PXTables VALUES('en','BEEST1','ANSI','2000','',null,'2010-22-03 09:00:00','2013-22-02 09:00:00','','','Annually','',null,null,0,null,null,'','','BE','Population','','',null,'Population January 1st by residence, gender, age and place of birth 1977-2013','','Population','Persons','','2013-22-02 09:00:00','','','','','Lars Pedersen, LARP@stat.gl',null,null,null,'Statistics Greenland','','',null,null,null,null,'','','','','','','','','',null,'','','',null,null,null)INSERT INTO statbank2013..PXVariables VALUES('en','BEEST1','time','1','R','','',null,'','',null,null,null,'','',null,'',null,null,'','')INSERT INTO statbank2013..PXValues VALUES('en','BEEST1','time','1','1977','1','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1978','2','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1979','3','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1980','4','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1981','5','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1982','6','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1983','7','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1984','8','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1985','9','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1986','10','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1987','11','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1988','12','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1989','13','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1990','14','',null,null,'','','','','',null,null,null,null,null,'',''),('en','BEEST1','time','1','1991','15','',null,null,'','','','','',null,null,null,null,null,'',''),

output with switch -s –o3 -e1

Converts our 574 Mb px-files to tab-separated csv-files

in less than

1 hour

Empty database (drop if exsist)Create 3 tables (tables, variables and values)

Loop though the px-files

set metafile=objf.opentextfile(f.name)uttext = metafile.readallmetafile.closeadoConn.execute uttextset metafile=nothing

Create table for data by use of newly added meta

Create insert statement for dataBULK INSERT statbank2013..PXData_NRE13 FROM 'G:\web\database\Greenland\National accounts\NRE13_data.sql' WITH (CODEPAGE = 'ACP',FIELDTERMINATOR='\t',ROWTERMINATOR='\n')

transfer script (load all)

simplePXsql (data tables)

simplePXsql (tables)

simplePXsql (variables)

simplePXsql (values)

what is that good for :

• a dirty px-file datacollection

• Used for cleansing• queries across tables• translation services• bridge to the nordic model ?

connections

PX-files

DBNordic

PX-webPC-Axis

PC-Axis/sqlDenmarkNorwayIreland…

Createprocedures

Createprocedures

Createprocedures

Createprocedures

simplePXsql

PX-Edit

nordic db model

• http://bank.stat.gl/m.asp

• Find table– a link (http://bank.stat.gl/m.asp?m=beest1) – category– search

• Table selected– known selection– unknown selection

px2go

• Greenland Tourism ( www.tourismstat.gl )

• automatic updated from our statbank

• api

s… happens

• Graph• Table• Pyramid• Bubles

http://json-stat.org

matrix Json-stat

getmeta

getdefault

showcube Graph

Table

explore

generate

API

Json-stat

csv

re-doing px2go for json-stat

• groupings

• production

cliffhanger

Screenshots from live - demo