Post on 25-Aug-2020
transcript
1
Paper 317-2011
The Day the Server Said Tweet! William W. Viergever, Viergever & Associates, California
Daniël Kuiper, SAS Institute B.V., The Netherlands Koen Vyverman, SAS Institute B.V., The Netherlands
ABSTRACT It was an idyllic summer afternoon at the SAS® Netherlands Technical Support call center. The pervasive hum and buzz of problem-solving activity was punctuated by the shrill cries of woodland birds drifting in through the open windows. Then, a dormant screen sprang to life, and panicky error messages and warnings started scrolling by! A distant siren wailed. Support staff gathered swiftly around the screen. It was the MIS server tweeting messages of impending doom! A SAS® Portal guru sat down at a console, logged on to the server, diagnosed, and fixed the problem. Not five minutes later, the screen went black again and relative serenity returned. The Managing Director had not even noticed the portal’s hiccups. This paper shows how to use Twitter to let your mission-critical SAS systems tweet their warnings and errors straight to your local support team for immediate follow up, and thus helps to minimize downtime!
INTRODUCTION System administrators typically have a variety of tools at their disposal to monitor the health of the systems and applications which they are support and maintain for their business users. Some of these monitoring solutions involve physically inspecting a reporting and diagnostics console application. Others are centered around a mechanism that sends diagnostic e-mail messages to the administrators. SAS Platform administrators in particular have access to the SAS 9.2 Enterprise Business Intelligence Audit and Performance Measurement (EBIAPM) utilities package, which monitors a number of aspects of a SAS Business Analytics server environment that are critical to the health of the system: is the SAS OLAP Server running? Is the Portal up? Is SAS Web Report Studio working?
Whatever the monitoring tool being used however, the bottom-line is always that an administrator first needs to notice that something is out of kilter. They will then begin to investigate what's going on, and attempt to fix things. While this will work in some cases, a full-fledged SAS Business Analytics environment can be a complex thing, and troubleshooting will take time, perhaps to eventually lead to the conclusion that SAS Technical Support needs to be involved. Valuable time is lost, the problem may become aggravated, and a business critical system goes from being merely unstable to being truly unavailable.
In this paper we present a mechanism to dramatically shorten the time between a problem occurring — or merely threatening to occur! — and your local Support team being on the case and potentially resolving the situation before it becomes a real issue, and hence reduce or avoid downtime. The idea is in essence a simple one: a server in the SAS environment (hereafter: the Server) tweets real-time diagnostic information to a specific Twitter account. The Support team receive these tweets on a dedicated screen and take action as needed! The main advantage of tweeting rather than sending diagnostic e-mails to the Support center is that the tweets are instantly visible whereas e-mails will sit in the incoming queue amidst a zillion other messages until they are read and processed by the operational dispatcher. Ideally, the two mechanisms complement one another. Bursts of tweets appearing on a monitor will attract Support staff attention, while an accompanying e-mail may contain more detail about the situation…
The first chapter of this paper explains what needs to be done on the Twitter side: setting up Twitter accounts for the Server and the Support center; ensuring that only the Support center is able to read the Server’s tweets; an introduction to OAuth, the authentication mechanism used by Twitter, and the various OAuth implementations available on the web. In the following chapter we choose a specific OAuth for Twitter implementation using PHP, and explain how to set up the PHP environment and scripts.
The next chapter explores three practical examples of Server-side SAS batch processes detecting certain worrisome events and tweeting about these: running out of disk space on the WORK volume; an important SAS server process going down; performance degradation by runaway user processes. We conclude the paper with some remarks about practicalities and suggestions for future development.
Throughout this paper we assume that both Server and client machines are running some Windows OS. On other operating systems, the necessary utilities will need to be replaced by suitable alternatives. The SAS code was developed and tested in a SAS 9.2 Maintenance 3 environment.
Social Media and NetworkingSAS Global Forum 2011
PREREQTHE BASTwitter is a140-characTwitter messonorous cTwitter site
So the Serhave unche
Tweet privathe Server to be used
QUISITES 1 SICS arguably the mocter messages ssage volume,cacophony. Wee (www.twitter.c
rver’s Twitter acecked all unne
acy is off by deand the Suppoby the Suppor
ost popular mic— ‘tweets’ — there are also
e want our Servcom) we sign u
Figure 1. C
ccount is SNLBcessary check
Fig
efault, and the sort center! We rt center: SNLT
R
cro-blogging se— out into the aeo quite a few apver to tweet to up for a new ac
Create the SNL
BAF, as in SASboxes. We log
gure 2. SNLBA
setting (Fig. 2) proceed to log
TECHSUPP. (F
2
ervice nowadayether. And, judgpplications, botan audience of
ccount. (Fig. 1)
BAF Twitter ac
S Netherlands Bon to the Twitt
AF Twitter acco
to protect the toff from Twitte
Fig. 3)
ys. Hundreds oging from a curts, and assortedf one though, n
ccount for the S
Business Analyter site as SNL
ount settings.
tweets is vital ter, and in simila
of millions of hursory examinatd automatons cnot to the entire
Server.
ytics FramewoLBAF and edit t
to keeping thinar vein sign up
umans use it totion of the dailycontributing to e world. On the
rk. Note that wthe account se
gs private betwfor another ac
o send y the
e
we ttings:
ween ccount
Social Media and NetworkingSAS Global Forum 2011
Just like wewhile settinjust want afollow, and
Logging onnow has 1
To display (www.tweeapplication
There is noSNLTECH
We then lo
Figu
e did for SNLBng up the accoua bare account. send a reques
Fi
n again as SNLfollower, viz. S
the Server’s twetdeck.com) Aft offers to sign
o need to creatSUPP Twitter a
og on to the Tw
re 3. Create th
AF, we log on unts, it is possi While logged st to follow SNL
Fig
igure 5. SNLBA
LBAF we see thSNLTECHSUPP
weets at the Suter installing Twin with a Twee
te a TweetDeckaccount creden
witter site as SN
e SNLTECHSU
to the Twitter sible to define inon as SNLTECLBAF. (Fig. 4)
gure 4. Send a
AF has a pendi
hat there is a pP.
upport center, wweetDeck on atDeck account
k account. We ntials. (Fig. 7)
NLBAF and sen
3
UPP Twitter ac
site as SNLTECnterests and imCHSUPP, we s
follow-request
ing follow reque
ending request
we deployed a Support cente, or alternative
simply click the
nd a test tweet.
ccount for the S
CHSUPP and amport friends. Nsearch for SNLB
t to SNLBAF.
est from SNLT
t. (Fig. 5) After
freeware applier machine andly, to add a soc
e ‘Add Twitter’
(Fig. 8)
Support center.
activate TweetNo need for all tBAF whose twe
TECHSUPP.
r accepting the
ication called Td launching it focial network ac
button and pro
.
Privacy. Note that however, weets we want t
request, SNLB
TweetDeck. or the first timeccount. (Fig. 6)
oceed to enter
that we o
BAF
e, the
the
Social Media and NetworkingSAS Global Forum 2011
The link is good. But rTwitter site
made! (Fig. 9) remember that
e. Which leads
Figure
Whatever SNLwe need SNLBus to the matte
Figure 6. Upo
e 7. Adding a T
Figure 8. A te
LBAF tweets isBAF to generaer of how Twitte
4
on running Twe
Twitter account
est tweet from S
s instantly displte its tweets ther handles auth
eetDeck.
t to TweetDeck
SNLBAF.
ayed in SNLTEhrough code, anhentication…
k.
ECHSUPP’s Twnd not by typin
weetDeck. So g stuff into the
far, so
Social Media and NetworkingSAS Global Forum 2011
OAUTH Until the mwasn’t veryHTTP. Sinc(www.oaut
For those kaccessible permit appsecrets are
Somewhatthe ApplicaTwitter usewe’ve namauthenticatit will use th
Clever, huhOAuth flowthe HMAC-SHA1 is a concepts fu
Luckily we have done libraries in these, keepthe chosenon a PHP-b
But before client via hSNLBIRDIEsigned in wname SNL
Twitter recobrowser soof code runchoose ‘cli
Figure
middle of 2010, y secure, but ace then howevth.net) which is
keen on undersfrom the aforelications to act
e being exchan
t oversimplifiedation) to be ableer SNLBAF. In
med SNLBIRDIEte itself: an OAhese, and there
h? There is mow diagram desc-SHA1 messagsecure hash aurther, any atte
don’t need to cthis before altha variety of lanping in mind th
n variety shouldbased solution
we get to the Pttp://twitter.comE is going to be
with). The fact tBIRDIE, and m
ognizes two apo we pick ‘cliennning in batch oent’, the registr
e 9. SNLTECH
Twitter used a t least it was eaer, Twitter has a different bea
standing OAuthementioned site
on their behalfnged.
perhaps, but we to send tweeorder to do tha
E) to pretend thAuth token, an Oe’s no need for
ore to it of courscribing the sevege authenticatiolgorithm well-kempt at explain
code SHA-1 enhough not in SAnguages: C/C+at the authenti
d integrate withand a couple o
PHP part of oum/oauth_clientse given permisthat the applica
make some cho
pplication typest’ as the applicon some serveration form insi
HSUPP can see
very basic autasy to send a tditched Basic
ast altogether.
h in its full glorye. Suffice it to sf without actua
what happens ets. That is, theat, the SNLBAFhat it is SNLBAOAuth token ser the Applicatio
se. The Twitteren steps involveon code to guanown in cryptoing them is wa
ncryption and HAS. The Twitte+, JAVA, Perl, cation in itself
h as little fuss aof really easy P
r set-up, we nes/new (Fig. 10) sion to act on t
ation itself is at oices.
s, client and brocation type. An er without any fists on an appl
5
e the test tweet
hentication metweet from a piAuth in favour
y, we refer to thsay that it is an ally providing th
is essentially thSAS program
F Twitter user teAF. Twitter thenecret, and a PIon to be aware
r developers sited in authentic
arantee the inteography. While ay out of scope
HMAC messager Developers s
PHP, Python, is not one of th
as possible withPHP-scripts by
eed to register This is where
the behalf of ththis point none
owser (Fig. 11)odd choice perront-end, but thication website
t from SNLBAF
echanism (Basiiece of SAS coof the open au
he Beginner’s G authentication
he user’s crede
his: we need a must use the T
ells Twitter than gives the AppN code. Wheneof the actual S
te (see Recomcation. Furthermegrity and authewe encourage for this paper…
ge authenticatiosite lists a slewRuby, you nam
he major pointsh our applicatioy one Joe Chun
our tweet-sendwe tell Twitter
he Twitter user existent is irrele
). Our SAS prorhaps in view ohe right choice e, so we enter a
F in TweetDeck
ic Auth) and lifeode simply by muthorization sta
Guide to OAuthn mechanism thentials. Instead,
SAS program Twitter servicet it is OK for th
plication a buncever the Applic
SNLBAF creden
mmended Readmore, the OAutenticity of trans
e the reader to e…
on from scratchw of publicly avame it. We invess of interest for ons written in Sng.
ding applicationthat an applicaSNLBAF (whic
evant. We mer
ocesses are noof the fact that nonetheless. Ea popular one j
k.
e was easy. It ameans of PROCandard OAuth
h which is hat allows user, tokens and to
(in OAuth spea on behalf of the application (w
ch of stuff to cation needs tontials.
ding) has a greath protocol reliesmitted messagexplore these
h, because otheailable OAuth stigated a few othis paper, and
SAS code. We s
n as a Twitter Oation named ch we are presrely register it b
t going to run fthey’ll merely bEven though wjust to comply.
also C
rs to oken-
ak: he which
o tweet
at es on ges.
ers
of d that settled
OAuth
sently by its
from a be bits
we We
Social Media and NetworkingSAS Global Forum 2011
also set thePOST requ
With theseread and wassociated
C
C
We take nointeracting Provider. Smust give t5849 OAut
Note (Fig. applicationkey and sepresses…
e default accesuest methods in
choices we gewrite access. Th with the SNLB
onsumer Key:
onsumer Secre
ote of these sinwith a Service
So the Consumto Twitter to proth specification
12) that it is po. This amounts
ecret in this pap
ss type to ‘Rean its interaction
Figure 10.
Figure 11. Ad
et a page (Fig. he confirmationBIRDIE applica
ilHsE6CDOCQ
et: pfaGMqejR
nce we will nee Provider on be
mer Key and theove its identity. (see Recomm
ossible to reset s to asking Twiper for didactica
d & Write’, whins with the Twit
The Twitter OA
dditional choice
12) confirmingn page also givation:
QR9tZ5sJUxQ
62un72kG3zc6
ed them in the Pehalf of a Usere Consumer Se. For more deta
mended Readin
the Consumertter for a new aal purposes sin
6
ch will enable Stter API.
Auth Applicatio
es to make at A
that the applicves us two long
6ogEh7VmC8P
PHP scripts. Inr. A User is the ecret are really ails on OAuth tg).
r Key and Consapplication ID ance we will hav
SNLBIRDIE to
on Registration
Application Reg
cation SNLBIRg alphanumeric
Pq3pyUD8Zldr4
n OAuth speak, individual who
y the credentialsterminology and
sumer Secret aand password, ve reset everyth
use both HTT
Form.
gistration
DIE was createc strings, which
4
a Consumer iso has an accous that the applid definitions we
associated withand therefore
hing by the tim
P GET and HT
ed by SNLBAFh are apparently
s an Applicatiount with the Serication SNLBIRe refer to the R
h the SNLBIRDit is safe to pue the paper hit
TTP
F with y
on rvice RDIE RFC
DIE blish ts the
Social Media and NetworkingSAS Global Forum 2011
Figur
PREREQTHE SOFAs explainebetween ourunning. ThWindows. Wour SAS sevariable.
The PHP pavailable o
To install Pa web servbut go with e.g. the OFor the pur
JOE CHUWhile seartutorial writbare-bonescommunitywe get five
The scripts
re 12. Twitter A
QUISITES 2 FTWARE ed in the previour SAS code ahe downloads sWe downloadeerver environm
project site onlyon the web, see
PHP, simply clicver’. The rest ofh the suggestedracle extensionrpose of this pa
UNG’S SCRIPrching for a suittten by Joe Chs PHP scripts ty at https://githue scripts and a f
s need some m
Application Reg
– PHP
ous chapter, wend Twitter. So section of the P
ed the php-5.3.ment. We then a
y offers 32-bit ce e.g. www.anin
ck through the f the installatiod extensions. An needs certainaper nothing be
PTS table OAuth to ung: http://nullito go with his tuub.com/joechunfew accompan
Figu
minimal editing.
gistration Confi
e plan to use awe need a PH
PHP project sit5-nts-Win32-V
added the locat
compiled Windondya.com
installer and wn is straightforw
Adding more exn Oracle client eyond the defa
Twitter impleminfo.wordpressutorial. These cng/oauth_twitteying files. (Fig.
ure 13. Joe Ch
We first edit th
7
rmation with O
an extant set ofP interpreter oe (www.php.neC9-x86.msi instion of the php.
ows binaries. F
when presentedward. A word oxtensions maysoftware to be ult component
mentation, we c.com/oauth-twican be downloaer After unzippi. 13)
hung’s OAuth T
he globals.php
OAuth Consume
f PHP scripts ton the machine et) offers a numstaller and so in.exe to the Win
For those need
d with a web seof warning howintroduce depepresent — and selection is ne
came across thitter/ Joe also paded from the ing the archive
Twitter scripts.
script, which c
er Key and Con
o handle the Owhere the SAS
mber of compilenstalled the PHndows PATH sy
ding a 64-bit bin
erver choice, chwever: do not peendencies on 3d PHP will refueeded.
he following maprovides a set GitHub open s
e to a suitable P
contains a num
nsumer Secret
OAuth authenticS code will be ed binaries for HP 5.3.5 releasystem environm
nary: there are
hoose ‘Do not serform a full ins3rd party softwause to even sta
arvelously lucidof self-containe
source code shPHP scripts dir
ber of definition
t.
cation
se on ment
some
set up stall are —rt up.
d ed aring
rectory
ns
Social Media and NetworkingSAS Global Forum 2011
used in thein the prev
// Fildefinedefine
Since we aThe readerof the DO_
CURLOPCURLOPCURLOP
In order to work. The and feed g
The Get Re(oauth_tok
R
R
ht
Note that thAuthorizati
Clicking the
P
We ignore the PIN… W
e other scripts. ious section:
ll in the ne('OAUTH_COe('OAUTH_CO
are behind a wer should provid_POST function
PT_PROXY =>PT_PROXYPORPT_PROXYUSE
authorize our SGet Request Tetreqtok.php to
equest Token sken_secret), an
Request Token
Request Token
ttp://api.twitter.
he URL containon API. Upon c
e ‘allow’ button
IN = 6582183
the instruction We will need th
Near the top o
next two conONSUMER_KEY'ONSUMER_SECR
eb proxy we alse their own pro
ns:
> 'your.proxRT => 80 ERPWD => 'pr
SNLBIRDIE apToken script (geo the PHP exec
Figure 1
script returns thd a URL:
= tew9CNU8xe
Secret = GhEK
com/oauth/aut&oauth_to&oauth_c
ns the Requestcopying and pa
Figure
n, we are rewar
to return to thehe PIN though
of the script we
nstants ', 'ilHsE6CRET', 'pfaG
so need to addoxy name, port
xy.here'
roxy_uid:pr
pplication to accetreqtok.php) ccutable. (Fig. 1
4. Upon runnin
hree items, a R
epcBtgmxFxbL
KtGRjmULRAq
horize?oauth_token_secret=G
callback_confirm
t Token and thasting the URL
e 15. Talking to
rded with a PIN
e SNLBIRDIE ain the next and
8
provide our Co
DOCQR9tZ5sJMqejR62un72
d a few Curl opt, and user–pas
oxy_pwd'
cess the SNLBcan be submitte
4)
ng the Get Req
Request Token
LmNDUkoQpCP
uTAoUehO5x4
token=tew9CNGhEKtGRjmULmed=true
e Request Tokin a browser w
o the Twitter Au
N-code (Fig. 16
application to ed final step of s
onsumer Key a
JUxQ'); 2kG3zc6ogEh7
tions to allow cssword, both in
BAF Twitter acced without edits
quest Token sc
(oauth_token)
PMmv904aglzw
4poVpDePuVF
NU8xepcBtgmxLRAquTAoUehO
ken Secret, andwe are asked to
uthorization AP
6):
enter the PIN, bsetting up our O
and Consumer
7VmC8Pq3pyU
connecting to thn the definition
count we need s. We open a c
cript.
), a Request To
w
FgSBgZMg
xFxbLmNDUkoO5x4poVpDeP
d feeds these to confirm acce
PI.
because there OAuth connecti
Secret as obta
UD8Zldr4');
he outside worof the DO_GE
to do some mocommand wind
oken Secret
QpCPMmv904PuVFgSBgZMg
to the Twitter ss. (Fig. 15)
is nowhere to ivity.
ained
rld. T and
ore ow
4aglzw g
enter
Social Media and NetworkingSAS Global Forum 2011
The last steWe edit theSecret, and
// Fil$reque$reque$oauth
We procee
The Get Ac(oauth_tok
A
A
Note that thall of your Anot currentindefinitely
So let’s put(tweet.phpmessage to
// Fil$acces$acces$tweet
Upon feedi
ep in preparinge Get Access Td PIN code:
ll in the nest_token='est_token_sh_verifier=
ed to feed the s
ccess Token scken), and an Ac
ccess Token =
ccess Token S
he Access TokAPI calls” indictly expire Accesy. Or at least un
t everything we). We edit the so be tweeted.:
ll in the nss_token='2ss_token_set = 'Frabjo
ing the tweet.p
Figure 16
g our Twitter OAToken script (ge
next 3 variatew9CNU8xep
secret='GhEK= 6582183;
script to PHP. (
Figure 1
cript returns theccess Token Se
= 256076484-3c
Secret = a2nS1
ken starts with acates the end oss Tokens andntil Twitter chan
e did so far to tscript to contain
next 2 varia256076484-3cecret='a2nS1ous day!';
php script to PH
6. The SNLBIRD
Auth connectivetacctok.php) b
ables. pcBtgmxFxbLKtGRjmULRAq
Fig. 17)
17. Upon runni
e two essentiaecret (oauth_to
cKiB4ae5saibK
wLtL9YtIk1me
a user ID. The of all the manua Access Token
nges its policy…
the test and seen our SNLBAF
ables. cKiB4ae5sai1wLtL9YtIk1
HP we get a loa
9
DIE application
vity is to requesby inserting ou
mNDUkoQpCPMuTAoUehO5x4
ng the Get Acc
l items we’ll neoken_secret):
K7TXYQAOvm
CqozY4q0LW0
phrase “Use thal steps which wn Secrets. The …
e whether we cAccess Token
bK7TXYQAOvmmeCqozY4q0L
ad of details ba
n is now author
st an Access Tor newly obtaine
Mmv904aglzw'4poVpDePuVFg
cess Token scr
eed to start twe
FcDMTF1bCw
01ECHsO9agG
he new oauth_we had to go thones which we
can now use Jon and Access T
mFcDMTF1bCwXLW01ECHsO9ag
ack from the Tw
rized.
oken and Acceed Request To
'; gSBgZMg';
ript.
eeting: an Acce
wX153orU
GK0kQ
_token and oauhrough. And ine just obtained
oe Chung’s TwToken Secret, a
X153orU'; gGK0kQ';
witter API. (Fig.
ess Token Secoken, Request T
ess Token
th_token_secrdeed, Twitter d should remain
weet script and provide the
. 18)
ret. Token
ret for does n good
e
Social Media and NetworkingSAS Global Forum 2011
And indeedthe ‘via SN
d, over at the SNLBIRDIE’ footn
F
Support center,note, indicating
igure 18. Upon
the TweetDecg that an applic
10
n running the Tw
ck shows the Scation SNLBIRD
Tweet script.
erver’s first scrDIE sent the tw
ript-generated weet on behalf o
tweet! (Fig. 19of SNLBAF.
) Note
Social Media and NetworkingSAS Global Forum 2011
APPLICANow that wfollowing thtweet.php stweet_tempelegance ocoding stan
TOO MUCIn our first gigantic da90% full, therror. The
option %let m%let t%let s%let d%let w filenafilenafilena data _ leng cm wo work cmd= callrun;
Figure 19. SNL
ATIONS we can send twhree examples script to send cplate.php. The or performancendards, nor wit
CH WORK! example we try
ata set by a zillihe Support stafSAS batch job
ns noxwait;
max_bytes threshold_pscripts_dirdir_out worklib
ame dirout ame tweettmame tweetmo
_null_; gth md $ 100 ork_volume k_volume=su="dir"||" "l system(cm
LTECHSUPP c
weets by callingare derived fro
custom messagreader should , but rather for th ours for that
y to detect the on variables atff can hopefullyto keep an eye
= 250048pct = 90; r = c:\whe = c:\tem = %sysfu
"&dir_outmp "&scriptsod "&scripts
$ 3; ubstr(symget||work_volu
md);
can see the scr
a script, the poom frequently rges in each casalso keep in mconceptual simmatter…
WORK library t once. By definy take action bee on WORK-sp
8507904;
erever\the\mp\dir_out.unc(getopti
t"; s_dir\tweets_dir\tweet
t("worklib"ume||" > &d
11
ript-generated t
ossibilities for sreported SAS Tse, and for rea
mind that the complicity. That is
running out of ning a suitable efore the entirepace could look
php\scriptstxt; on(work));
_template.p.php";
),1,3); ir_out.";
test tweet from
sending real-timTechnical Suppasons of code rode presented s, it’ll do the job
disk space, e.g threshold, say
e environment gk like this:
s\are;
php";
m SNLBAF in Tw
me diagnosticsport issues. Wereadability we rhere has not b
b but it may no
g. because a uy the WORK vogrinds to a halt
weetDeck.
s are endless. Te will use the rename the scrbeen optimizedt comply with y
user is trying toolume running ot on a ‘Disk Ful
The
ript to for
your
o sort a over ll’
Social Media and NetworkingSAS Global Forum 2011
12
data foo; infile dirout truncover; input line $60.; run; data _null_; length from to positions bytes_free pct_full 8; set foo end=last; if last then do; from=find(line,')')+2; to=find(line,'bytes'); positions=to-from; bytes_free=input(compress(substr(line,from,positions),','),20.); pct_full=round(100-bytes_free/"&max_bytes"*100); call symput('pct_full',trim(left(put(pct_full,2.)))); end; run; data _null_; length bytes 8 tweet_msg $ 140; retain tweet_msg; if _n_=1 then tweet_msg="$tweet = 'the WORK volume is "||"&pct_full" ||"% full - %sysfunc(datetime(),datetime19.)';"; infile tweettmp; file tweetmod; input; if not (_infile_=:"$tweet") then put _infile_; else put tweet_msg; if &pct_full < &threshold_pct then call symput('tweetflag','0'); else call symput('tweetflag','1'); run; data _null_; if &tweetflag=1 then call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;
A few comments about the code:
We will interact with the OS through a DOS command window. Setting the NOXWAIT option allows the SAS code to continue running instead of waiting for a human to close the command window.
Some global variables and assorted definitions: Max_bytes is the total storage capacity of the volume where the WORK library lives, expressed in bytes; Threshold_pct is the volume usage percentage above which we want to send a warning tweet; Scripts_dir is the directory where Joe Chung’s PHP scripts were saved; Dir_out points to a txt-file which will capture the output of a DIR command; Worklib is the directory path to the WORK library; Tweettmp points to the tweet_template.php script; Tweetmod points to the tweet.php script, which will be the modified copy of the template to be executed.
We extract the volume name from the WORK library path, and using CALL SYSTEM we run a dir command on it while piping the output thereof to the text-file dirout.
Read the output of the dir command and store as text strings in a temporary data set.
The last line of dir command output contains the number of free bytes on the volume, so we parse the last observation to pry the number (bytes_free) out of the text, and calculate the percentage full for the volume (pct_full).
Using pct_full we construct the actual tweet-message (tweet_msg), or rather the entire line in the tweet_template.php script that needs to be replaced to send a custom tweet. Note that we include a datetime stamp within the body of our tweet. We did this while developing and testing the code because Twitter imposes restrictions on tweeting the same message multiple times. We found it rather practical however since it shows the
Social Media and NetworkingSAS Global Forum 2011
tweet-ti
We thedefinedtweet_mmessag
If the pethen theotherwi
If the tw
When the t
ANYBODIn this examMeasuremcan be conThis is dongenerates
option %let s%let e%let x libnamfilenafilenafilena data _ file put put run; x &xcm
ime quite visibl
en read the twed, we simply oumsg we construge substituting
ercentage full fe tweetflag is sise nothing sho
weetflag is set,
threshold is pa
Figure 20. The
DY HOME? mple we use daent (EBIAPM)
nfigured to run ne by schedulinthe sas92_bi_h
ns noxwait;
scripts_direbiapm_homexcmds
me apmdataame xcmds ame tweettmame tweetmo
_null_; e xcmds; "cd &ebiap'runchecks
mds;
y in the Tweet
et_template.phutput the _INFILucted earlier. Tthe default one
for the volume set to 1. For theows.
we call PHP a
ssed, a tweet i
e SNLTECHSU
ata generated package as refregular checks
ng the runcheckhealth data set
r = c:\wheree = c:\sas\e = c:\temp\
a "&ebiapm_ "&xcmds";
mp "&scriptsod "&scripts
pm"; server.bat';
Deck app, so w
hp script line byLE_ buffer agaThe result is a ce.
is below the the purpose of th
nd feed it our c
s sent and it sh
UPP TweetDec
by the SAS 9.2ferenced in the
s on a whole lotkserver.bat (wht, which we ins
ever\the\phebiedieg\le\xcmds.bat;
_home\data\; s_dir\tweets_dir\tweet
;
13
we decided to l
y line. Unless tin. For the line copy of the twe
hreshold we sethis demo we ha
customized twe
hows up on the
ck shows the w
2 Enterprise Bue introduction. Wt of things. Likehich comes witpect in the follo
p\scripts\av1\ebiapm92
status";
_template.p.php";
eave it in.
he line is the o commencing ‘
eet_template.ph
t a tweetflag toave temporarily
eet.php script.
e Support cente
warning tweet fr
usiness IntelligWhen a SAS ae whether any h EBIAPM) jobowing code:
are; 2;
php";
one where the b‘$tweet’ we subhp script but w
o zero. If it excey lowered the th
er’s TweetDec
rom the SNLBA
ence Audit andadministrator inof the vital SAS
b to run periodi
body of the twebstitute the
with our own
eeds the threshhreshold to 70%
k. (Fig. 20)
AF server.
d Performancenstalls EBIAPMS Servers are dcally. The batc
eet is
hold %
, it down.
ch job
Social Media and NetworkingSAS Global Forum 2011
14
data servers_down; set apmdata.sas92_bi_health; if server_status=0 then output; run; data _null_; dsid=open("servers_down"); nrobs=attrn(dsid,"nobs"); rc=close(dsid); if nrobs=0 then call symput('tweetflag','0'); else do; infile tweettmp end=last; file tweetmod; input; if not (_infile_=:"$tweet") then put _infile_; else put "$tweet = 'One or more SAS Servers are down - %sysfunc(datetime(),datetime19.)';"; if last then call symput('tweetflag','1'); end; run; data _null_; if &tweetflag=1 then call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;
The structure of the code is similar to the previous example. Items that were already explained or commented upon will not be reiterated here:
Some global variables and assorted definitions: Ebiapm_home is the home directory of our EBIAPM installation; Xcmds points to a text-file which we’ll use to run a few X commands in one go; Apmdata points to the location where the EBIAPM runcheckserver.bat job writes its data sets;
The OS commands to run the EBIAPM runcheckserver.bat job are written to a text-file and subsequently executed by means of an X statement. Note that ordinarily the frequency of running this particular job should be kept as low as possible due to the potential impact on system performance.
We then inspect the output of the runcheckserver.bat process — the data set sas92_bi_health — and retain only those records where a server down condition was detected.
If any servers are found to be down we create a custom tweet.php from the tweet_template.php script by inserting an appropriate tweet message in the right place.
And run the script if necessary…
An obvious improvement to the above would be to send more specific tweets based on the contents of the servers_down data set, explicitly mentioning in the tweet body which servers are impacted. For now, figure 21 shows the generic warning popping up in the Support center’s TweetDeck.
Social Media and NetworkingSAS Global Forum 2011
BOGGEDAs a final eprocesses environmereference ssessions th
The standarun as the Pooled Wooutput of aindividual u
In order to previously us/sysinter
The followi
option %let s%let p%let x filenafilenafilenafilenafilena data _ file put put put run; x &xcm
Figure 2
D DOWN example we attgobbling up Cnt is configuredsystem howevehat are typically
ard set-up on W‘NT AUTHORI
orkspace Serve DOS tasklist c
users’ accounts
monitor how loby SysInternal
rnals/bb896649
ing code detec
ns noxwait;
scripts_dirpstools_homxcmds
ame pslist ame tsklistame tweettmame tweetmoame xcmds
_null_; e xcmds; "cd &pstoo"pslist sa"tasklist
mds;
21. Another om
tempt to detectPU cycles and d and used it mer, it is not. So y active in a SA
Windows is thatTY\SYSTEM’ u
er is running ascommand, any s.
ong processes s, nowadays to
9
ts user Worksp
r = c:\whereme= c:\pstoo = c:\temp\
'c:\temp\t 'c:\temp\mp "&scriptsod "&scripts "&xcmds";
ols_home"; as > c:\temp/V /FI ""im
minous warning
t system perforbogging down
may be perfectlyhow do we det
AS environmen
t the Metadata user. Furtherms the SAS GeneSAS.exe proce
have been runo be found on t
pace Server pro
ever\the\phols; \xcmds.bat;
\pslist.txt\tasklist.ts_dir\tweets_dir\tweet;
p\pslist.txmagename eq
15
g shows up in th
mance degradn the entire systy legitimate to tect the relevannt?
Server, OLAPore, everythingeral Server Useesses left are W
nning we use ththe MS site: htt
ocesses runnin
p\scripts\a
'; xt'; _template.p.php";
t"; sas.exe""
he Support cen
ation by hangintem. Dependinhave long-runnnt SAS.EXE pr
P Server, Conneg running on aer (SASSRV). Workspace Se
he PsList utilitytp://technet.mic
ng for more tha
are;
php";
> c:\temp\t
nter’s TweetDe
ng user Worksng on how exacning user procerocesses amon
ect Server, andStored ProcesSo if we filter trver processes
y which is part ocrosoft.com/en
an 2 days:
tasklist.tx
eck.
space Server ctly the SAS esses. In our ng the mass of
d Share Serverss Server and athose out of thes running unde
of the PsTools -
xt";
SAS
r all a e r the
suite,
Social Media and NetworkingSAS Global Forum 2011
16
data pslist; infile pslist firstobs=4; input name $ pid pri thd phd priv @49 cpu_time time11. @65 elapsed_time time11.; run; data tsklist; infile tsklist firstobs=4; input name $ pid system $ session memory comma10. kilobyte $ status $ user $40.; run; proc sql; create table processes as select tsklist.user, pslist.elapsed_time from pslist, tsklist where pslist.pid=tsklist.pid ; quit; data user_processes; set processes; if upcase(user) in ("NT AUTHORITY\SYSTEM","FOO\SASSRV") then delete; run; proc sort data=user_processes; by descending elapsed_time; run; data _null_; dsid=open("user_processes"); nrobs=attrn(dsid,"nobs"); rc=close(dsid); if nrobs=0 then abort return; run; data _null_; set user_processes(obs=1); if elapsed_time<172800 then abort return; run; data _null_; infile tweettmp end=last; file tweetmod; input; if not (_infile_ =: "$tweet") then put _infile_; else put "$tweet='One or more Workspace Server user processes is running longer than 2 days! - %sysfunc(datetime(),datetime19.) ';"; run; data _null_; call system('php'||" "||"&scripts_dir"||'\tweet.php'); run;
Some additional notes:
Some global variables and assorted definitions: Pstools_home is the home directory of our PsTools installation;
Social Media and NetworkingSAS Global Forum 2011
Xcmds Pslist pTsklist
In the xredirectsas.exe
After ruinto a p(PID), wavailab
We the
And filte
Then soobserva
If the resession
If the loalso ba
We onldays. W
Figure 22 s
CONCLUWe have senvironmeadequatelyHowever, imechanismTwitter by ucan be donA fun exerc
points to a texpoints to a temppoints to a tem
xcmds.bat file wt the output to te processes, an
unning these copair of data setswhereas the tskle in the text-fil
en join the user
er out anything
ort by descendation.
esulting data sen, preventing th
ongest running ail out.
y get to this paWe modify the t
shows the code
Figure 22
USION hown how to snt to tweet diagy protected agan case someth
m should providusing Joe Chune through SAScise!
xt-file which weporary text-file
mporary text-file
we first move tothe pslist.txt filend redirect the
ommands in bas. The pslist daklist data set wles, it’s interest
name to the e
g that is not run
ding elapsed tim
et is empty we he rest of the co
user process h
art of the code itweet message
e-generated tw
. Another warn
set up a mechagnostic messagainst downtime hing goes awryde some additiong’s PHP scripS code. To mak
’ll use to run a wherein we ca
e for capturing t
o our PsTools he. We also calloutput to the t
atch with an X sata set will essewill link PIDs to ting to see wha
lapsed process
nning under an
me, so that the
don’t need to dode being exec
has an elapsed
if there is at leae as in the prev
weet in the Sup
ing is tweeted
nism allowing Sges to a suppoby solid failove
, it is always goonal peace of mpts, but it shouldke things intere
17
few X commanpture the outputhe output of a
home directory the tasklist coasklist.txt file.
statement, we entially give ususers. For dida
at other proces
s time on PID.
individual user
longest runnin
do anything elscuted.
d time of less th
ast one user prvious examples
port center’s Tw
to the Support
SAS processesort center. Businer mechanismsood to involve tmind for SAS Pd be interestingesting, this wou
nds in one go;ut of a PsList cTasklist comm
y, then call the ommand with op
proceed to reas the elapsed timactical purposess-related inform
r account.
ng user process
se and ABORT
han 2 days (ex
rocess with an s, and call PHP
weetDeck.
t center’s Twee
s running in a Sness critical sys at all levels: dthe experts as
Platform adming to see whethuld require cod
command; mand.
PsList tool for ptions for verbo
ad the newly geme for each sa
es we also readmation is in the
s is represente
T RETURN kills
pressed here i
elapsed time oP to send it.
etDeck for follo
SAS Business ystems should odatabase, hardquickly as posistrators. We to
her the entire Oing SHA1 encr
SAS processeosity and filteri
enerated text fias.exe processd the other fieldere.
ed by the first
s the current SA
n seconds) the
of more than tw
w-up.
Analytics servof course be
dware, applicatssible. The propook a shortcut
OAuth authenticryption in Base
s and ng for
les ID
ds
AS
en we
wo
er
ions. posed to
cation e SAS.
Social Media and NetworkingSAS Global Forum 2011
18
RECOMMENDED READING • SAS 9.2 Enterprise Business Intelligence Audit and Performance Measurement for Windows Environments
http://support.sas.com/rnd/emi/EbiApm92/sas92.ebiapm.win.pdf
The installation and configuration guide for the EBIAPM package gives a good idea of the variety of SAS environment aspects that can be monitored by the utility.
• The Twitter Developers Authentication page http://dev.twitter.com/pages/auth
For those so inclined, we recommend having a look at the details of the OAuth authentication flow as explained on this site.
• RFC 5849 – The OAuth 1.0 Protocol http://tools.ietf.org/html/rfc5849
The complete RFC document describing version 1 of the OAuth protocol. This is hardcore.
ACKNOWLEDGEMENTS The authors should like to thank SAS Global Forum section chair Pete Lund for his interest in this somewhat maverick application of Social Media abuse in a SAS systems administration context, and the inimitable Joe Chung for making his excellent scripts publicly available and taking the bite out of OAuth!
CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the authors at:
William W. Viergever Viergever & Associates Health Data Analysis / Systems Design & Development 2920 Arden Way Suite N Sacramento, CA 95825 USA william@viergever.net www.viergever.net +1 (916) 483-8398 Daniël Kuiper / Koen Vyverman SAS Institute B.V. Flevolaan 69 1272PC Huizen The Netherlands support@snl.sas.com
SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.
Other brand and product names are trademarks of their respective companies.
Social Media and NetworkingSAS Global Forum 2011