Date post: | 28-Jan-2015 |
Category: |
Technology |
Upload: | gun-lee |
View: | 113 times |
Download: | 1 times |
The Glass Class: Mirror API
July 7th – 11th 2014
Mark Billinghurst, Gun Lee
HIT Lab NZ
University of Canterbury
An Introduction to
Glassware Development
Mirror API using Java Servlet
Gun Lee
* Images in the slides are from variety of sources,
including http://developer.android.com and http://developers.google.com/glass
Glassware Development
Mirror API
Server programming, online/web application
Static cards / timeline management
GDK
Android programming, Java (+ C/C++)
Live cards & Immersions
https://developers.google.com/glass/
Mirror API
REST API
Java servlet, PHP, Go,
Python, Ruby, .NET
Timeline based apps
Static cards
- Text, HTML, media attachment (image & video)
- Standard and custom menu items
Manage timeline
- Subscribe to timeline notifications
- Sharing with contacts
- Location based services
https://developers.google.com/glass/develop/mirror/index
Mirror API based Web App
3. Insert a static card
User sees the card
Glassware Web app
https://developers.google.com/glass/develop/mirror/stories
Develop with Mirror API
Prepare a web server
https callback required for OAuth 2.0
Minimum storage for credentials
Create a Google APIs Console project
Enable Mirror API and get Client ID & secret
Create a web application
Java servlet, PHP, Go, Python, Ruby, .NET
Implement OAuth 2.0 authorization
Use Mirror API to make REST API calls
- Wrapper classes/methods provided
Google App Engine
Free web server space!
You can skip this step if you have another server
Upgrade to paid service based on storage and traffic
needed
https://developers.google.com/appengine/
Python, Java, PHP, Go
Cloud Storage
Easy integration with Google APIs
Google APIs Console
http://console.developers.google.com
Login with your own Google account
Create a New Project
The Project ID becomes your URL
- http://your-project-id.appspot.com
Google APIs Console
In the project’s APIs & auth section,
enable Google Mirror API
Google APIs Console
APIs & auth > Credentials > OAuth: Create
New Client ID
Choose "Web Application"
Type in your website and OAuth callback
address https://your-app-id.appspot.com
https://your-app-id.appspot.com/oauth2callback
Google APIs Console
Note both Client ID and Client secret
Google APIs Console
APIs & auth > Consent screen
Fill in the Product Name and Homepage URL
This information is shown when users
authorizing your web app
Live Demo
- Setup Google App Engine
- Setup Google APIs Console
Dev. Env. Setup
ADT Bundle (Eclipse + Andriod SDK)
http://developer.android.com/sdk/index.html
GAE plugin for Eclipse
http://dl.google.com/eclipse/plugin/4.2
- Google App Engine Tools for Android
- Google Plugin for Eclipse
- SDK > Google App Engine Java SDK
Sign in on Eclipse GAE plugin
Use the Google account that will host your
web app on GAE
Create a Web App for GAE
File > New > (Other > Google > )
Web Application Project
Fill in the Project name and Java package
namespace to use
Check on option Use Google App Engine
Uncheck option Use Google Web Toolkit
Select option Use App Id then click Browse
button to choose your app id
Check on Generate project sample code
Click Finish
Anatomy of a Web App Project
Web App Project
src (source folder)
- Java classes organized in namespaces
war (web application archive)
- WEB-INF (settings and JAR libraries)
• appengine-web.xml
• web.xml
• lib
- index.html
- favicon.ico
Deploying to GAE
Right click on the Project
Google > Deploy to App Engine
Uncheck option Launch app in browser ...
Click Deploy
Open your GAE site on a browser
http://your-app-id.appspot.com
Live Demo
- Creating a new Web App project
- Anatomy of a Web App project
- Deploy to GAE
Adding Mirror API library
Right click on the Project
Google > Add Google APIs
Choose Google Mirror API
Click Finish
Authorization with OAuth 2.0 Authorizing your web app to access timeline on user’s
Google account
Programming Google Glass - The Mirror API by Eric Redmond
http://pragprog.com/book/erpgg/programming-google-glass
Authorization with OAuth 2.0
Enable HTTP sessions on GAE
Force https access only
Filter out inappropriate access
Implement OAuth 2.0 callback servlets
Code from Quick Start sample project
- https://developers.google.com/glass/develop/mirror
/quickstart/index
Add a new servlet to logout
https://developers.google.com/glass/develop/mirror/authorization
Enable sessions on GAE
war/WEB-INF/appengine-web.xml
<appengine-web-app ...>
...
<sessions-enabled>true</sessions-enabled>
</appengine-web-app>
Force https access only
war/WEB-INF/web.xml
<web-app ...>
...
<!-- force https only -->
<security-constraint>
<web-resource-collection>
<web-resource-name>Protected Area</web-resource-name>
<url-pattern>/*</url-pattern>
</web-resource-collection>
<user-data-constraint>
<transport-guarantee>CONFIDENTIAL</transport-guarantee>
</user-data-constraint>
</security-constraint>
...
</web-app>
Define Auth Filters war/WEB-INF/web.xml
<web-app ...>
...
<!-- filters -->
<filter>
<filter-name>authFilter</filter-name>
<filter-class>com.google.glassware.AuthFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>authFilter</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
<filter>
<filter-name>reauthFilter</filter-name>
<filter-class>com.google.glassware.ReauthFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>reauthFilter</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
</web-app>
Define Auth Servlet
war/WEB-INF/web.xml <webapp ...>
…
<servlet>
<servlet-name>oauth2callback</servlet-name>
<servlet-class>com.google.glassware.AuthServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>oauth2callback</servlet-name>
<url-pattern>/oauth2callback</url-pattern>
</servlet-mapping>
...
</webapp>
Add Java classes for Auth
Create com.google.glassware package in
the src folder
Right click on src folder > New > Package
Copy Java classes for Auth
Drag and drop the following .java files into the
created package
AuthFilter.java, AuthServlet.java,
AuthSettings.java, AuthUtil.java,
ReauthFilter.java, WebUtil.java
Add Java classes for Auth
In AuthSettings.java
Fill in your CLIENT_ID and CLIENT_SECRET
strings you’ve noted from Google APIs
Console’s credential section
public static String CLIENT_ID =
"567615950876-14k9b9sggrtpulhu9s72le4vsejjscak.apps.googleusercontent.com";
public static String CLIENT_SECRET = "lo9hajhpQFneXP5K8YR0gEVK";
Add a New Servlet for Logout
Add LogoutServlet class
Right click on your app package in src folder
New > Class
Name: LogoutServlet
Superclass: javax.servlet.http.HttpServlet
Add a New Servlet for Logout
Implement LogoutServlet class
Open LogoutServlet.java
Source > Override/Implement Methods ...
Check on doGet(…) the click OK
Call AuthUtil.clearUserId(req) in the method
// log out
AuthUtil.clearUserId(req);
// print out results on the web browser
resp.setContentType("text/html; charset=utf-8");
resp.getWriter().println(
"<html><head><meta http-equiv=\"refresh\" content=\"3;url=/index.html\"></head> " +
"<body>Good bye!<br></body></html>" );
Add a New Servlet for Logout
Add servelt tag to war/WEB-INF/web.xml <webapp ...>
…
<servlet>
<servlet-name>Logout</servlet-name>
<servlet-class>org.hitlabnz.hw.LogoutServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>Logout</servlet-name>
<url-pattern>/logout</url-pattern>
</servlet-mapping>
...
</webapp>
Add a New Servlet for Logout
Add a link to the LogoutServlet in
index.html
<a href="logout">Logout</a>
Live Demo
- Enable HTTP sessions on GAE
- Filter out inappropriate access
- Implement OAuth 2.0 callback
- Add logout servlet
Get Access to Mirror API
In HelloWorldServlet class
Add method getMirror(req)
private Mirror getMirror(HttpServletRequest req) throws IOException {
// get credential
Credential credential = AuthUtil.getCredential(req);
// build access to Mirror API
return new Mirror.Builder( new UrlFetchTransport(),
new JacksonFactory(), credential)
.setApplicationName("Hello Glass!")
.build();
}
Insert a Card to Timeline
In HelloWorldServlet class
Add code in doGet(...) method // get access to Mirror API
Mirror mirror = getMirror(req);
// get access to the timeline
Timeline timeline = mirror.timeline();
// create a timeline item (card)
TimelineItem timelineItem = new TimelineItem()
.setText( "Hello World!" )
.setDisplayTime(new DateTime(new Date()))
.setNotification(new NotificationConfig().setLevel("DEFAULT"));
// insert the card into the timeline
timeline.insert(timelineItem).execute();
Insert a Card to Timeline
In HelloWorldServlet class
Modify code in doGet(...) method for richer
response on the web browser
// print out results on the web browser
resp.setContentType("text/html; charset=utf-8");
resp.getWriter().println(
"<html><head>“ +
"<meta http-equiv=\"refresh\"content=\"3;url=/index.html\">" +
"</head> " +
"<body>A card is inserted to your timeline.<br></body></html>" );
Live Demo
- Access to Mirror API
- Insert a Card into Timeline
HTML in a Card
Follows CSS for the Glass UI https://mirror-api-playground.appspot.com/assets/css/base_style.css
String html = "<article><section><p class=\"text-auto-size\">" +
"<em class=\"yellow\">Hello World!</em><br>" +
"Welcome to the <strong class=\"blue\">Glass Class</strong> at HIT Lab NZ." +
"</p></section></article>";
timelineItem.setHtml(html);
Google Mirror API Playground
https://developers.google.com/glass/tools-
downloads/playground
To try it with your Glass
Go to Google API console
- https://console.developers.google.com
- Login and open your project
Create a new Client ID with the following URL
- https://mirror-api-playground.appspot.com
Use the generated Client ID to authorize
access on the Playground site
Multiple Pages in a Card
Auto paginate
Manually paginate
Multiple articles
<article class="auto-paginate">
Very long article ...
</article>
<article class="cover-only">
<section>
<p class="text-auto-size">This is the cover
page.</p>
</section>
</article>
<article>
<section>
This is second page.
</section>
</article>
<article>
<section>
Third page.
</section></article>
https://developers.google.com/glass/develop/mirror/timeline
Grouping Multiple Cards
Bundle – a set of cards grouped under a
cover card (or the latest card)
timelineItem.setBundleId( “UniqueBundleID” )
timelineItem.setIsBundleCover( true );
https://developers.google.com/glass/develop/mirror/timeline
Built-in Actions with Menu READ_ALOUD, DELETE, OPEN_URI, PLAY_VIDEO,
TOGGLE_PINNED, VOICE_CALL, NAVIGATE, REPLY,
SHARE
List<MenuItem> menuItemList = new ArrayList<MenuItem>();
menuItemList.add(new MenuItem().setAction("READ_ALOUD"));
timelineItem.setSpeakableText("Hello World!");
menuItemList.add(new MenuItem().setAction("TOGGLE_PINNED"));
menuItemList.add(new MenuItem().setAction("DELETE"));
timelineItem.setMenuItems(menuItemList);
https://developers.google.com/glass/develop/mirror/static-cards#creating_menu_items
Live Demo
- Cards with HTML
- Add Menu Items
Say hello to me
Add web form in index.html
Override doPost(...) method and get
parameter from the request
<form action="helloworld" method="post">
Hello <input type="text" name="custom_name" value="World"> !
<input type="submit" value="Submit">
</form>
String custom_name = req.getParameter("custom_name");
String message = "Hello " + custom_name = "!";
Image attachment
Adding static resources to the server
Edit war/WEB-INF/web.xml
Add a folder named static under the war
folder and add image files in it
<web-app ...>
...
<static-files>
<include path="/static/*" />
</static-files>
...
</web-app>
Image attachment
Attached image will be used as the
background of the card
In low-level HTTP Multipart attachment
// get an image to use with the card
URL url = new URL("http://hello-world-mirror.appspot.com/static/hitlabnz.jpg");
InputStreamContent attachment =
new InputStreamContent("image/jpeg", url.openStream());
// insert the card into the timeline
timeline.insert(timelineItem, attachment).execute();
https://developers.google.com/glass/develop/mirror/static-cards#inserting_a_timeline_item_with_media
Live Demo
- Input from web forms
- Image attachment
Managing Timeline Items
List
Delete
https://developers.google.com/glass/v1/reference/timeline/list
// request for list of the timeline items
Mirror.Timeline.List listRequest = timeline.list();
listRequest.setMaxResults(5L);
TimelineListResponse listResponse = listRequest.execute();
// retrieve results
List<TimelineItem> timelineItemList = listResponse.getItems();
mirror.timeline().delete(deleteItemId).execute();
Update Pinned Timeline Item
Simulating Live Cards with Mirror API
Ask the user to pin the item
Update the item regularly TimelineItem timelineItem = getSavedTimelineItem();
// If a saved item exists, isn't deleted, and is pinned
if (timelineItem != null
&& !(timelineItem.getIsDeleted() != null && timelineItem.getIsDeleted()) &&
(timelineItem.getIsPinned() != null && timelineItem.getIsPinned()))
{
// update the pinned item
timelineItem.setHtml( new_html );
timeline.update( timelineItem.getId(), timelineItem ).execute();
} else {
// create a new one and save the id for latter use
}
https://developers.google.com/glass/v1/reference/timeline/update#examples
Live Demo
- List Timeline Items
- Delete Timeline Items
Location
Get latest known location
Draw maps in your card
Location location = mirror.locations().get("latest").execute();
double latitude = location.getLatitude();
double longitude = location.getLongitude();
https://developers.google.com/glass/develop/mirror/location
<article>
<figure>
<img src="glass://map?w=240&h=360&marker=0;42.369590,-71.107132"
height="360" width="240">
</figure>
<section>
<div class="text-auto-size">
<p>Map shown on the left</p>
</div>
</section>
</article>
Navigate Menu Item
Set Location to the timeline item and add the
NAVIGATE menu item ...
timelineItem.setLocation(
new Location()
.setLatitude( -43.522087 )
.setLongitude( 172.582823 )
.setAddress("University of Canterbury, Ilam, Christchurch")
.setDisplayName("HIT Lab NZ") );
// add menu items with built-in actions
List<MenuItem> menuItemList = new ArrayList<MenuItem>();
menuItemList.add(new MenuItem().setAction("NAVIGATE"));
timelineItem.setMenuItems(menuItemList);
// insert the card to the timeline
timeline.insert(timelineItem).execute();
Live Demo
- Locate My Glass
- Add a Navigation Card
Subscriptions
Mirror API (server) calls back your web app with
notification POST
Your web app can subscribe to
Location updates
Timeline updates
- Share, reply, delete, custom menu item, launch with voice
Your web app must respond with a 200 OK
HTTP status code if no error occurred.
If not, Mirror API might resend notifications.
https://developers.google.com/glass/develop/mirror/static-cards#subscriptions
Subscribe
Add subscription to the mirror api
Better check if already subscribing
Subscription subscription = new Subscription();
subscription.setCollection("locations"); // or "timeline"
subscription.setCallbackUrl("https://your_web_app/notification");
subscription.setUserToken( AuthUtil.getUserId(req) );
mirror.subscriptions().insert(subscription).execute();
Unsubscribe
Add subscription to the mirror api // find subscription to delete
SubscriptionsListResponse subslist_resp =
mirror.subscriptions().list().execute();
List<Subscription> subsclist = subslist_resp.getItems();
String subscription_id_to_delete = null;
for (Subscription subsc : subsclist) {
if (subsc.getId().equals("locations")) { // or “timeline"
subscription_id_to_delete = subsc.getId();
break;
}
}
// delete the subscription
mirror.subscriptions().delete(subscription_id_to_delete).execute();
Handle Notification Callback
Create a callback Servlet that
handles notification POST
responds with a 200 OK HTTP status code
Make sure your callback Servlet URL is
not filtered in AuthFilter.java
Use the user ID provided by
notification.getUserToken() to call Mirror
APIs subsequently
Handle Notification Callback protected void doPost(HttpServletRequest req, HttpServletResponse resp) {
// Respond with OK and status 200 to prevent redelivery
resp.setContentType("text/html");
Writer writer = resp.getWriter();
writer.append("OK");
writer.close();
// extract notification object
JsonFactory jsonFactory = new JacksonFactory();
Notification notification =
jsonFactory.fromInputStream(req.getInputStream(), Notification.class);
// Figure out the impacted user and get their credentials for API calls
String userId = notification.getUserToken();
Credential credential = AuthUtil.getCredential(userId);
Mirror mirror = new Mirror.Builder(new UrlFetchTransport(),
new JacksonFactory(), credential)
.setApplicationName("Hello World")
.build();
if (notification.getCollection().equals("locations")) { // or “timeline“
...
Handle Notification Callback
Handling location updates
Handling timeline updates
User action type
- REPLY, DELETE, CUSTOM, LAUNCH, SHARE
// get notified location
Location location = mirror.locations().get(notification.getItemId()).execute();
location.getLatitude();
location.getLongitude();
...
// Get the timeline item which triggered notification
TimelineItem notifiedItem = mirror.timeline().get(notification.getItemId()).execute();
if(notification.getUserActions().contains(new UserAction().setType("REPLY")))
{
String message = "Hello " + notifiedItem.getText() + "!";
...
https://developers.google.com/glass/develop/mirror/subscriptions
Live Demo
- Subscribe to Location Updates
- Subscribe to Timeline Updates:
REPLY
Contact
LAUNCH and SHARE user actions need
contact of your web app
Contact helloWorldContact = new Contact();
helloWorldContact.setId("org.hitlabnz.helloworld");
helloWorldContact.setDisplayName("Hello World");
ArrayList<String> imageUrlList = new ArrayList<String>();
imageUrlList.add("http://hello-world-mirror.appspot.com/static/hitlabnz.jpg");
helloWorldContact.setImageUrls(imageUrlList);
ArrayList<Command> commandList = new ArrayList<Command>();
commandList.add(new Command().setType("POST_AN_UPDATE")); // TAKE_A_NOTE
helloWorldContact.setAcceptCommands(commandList);
https://developers.google.com/glass/develop/mirror/contacts
https://developers.google.com/glass/develop/mirror/subscriptions
Live Demo
- Subscribe to LAUNCH
DIY
- CUSTOM menu item
- Subscribe to SHARE
https://developers.google.com/glass/develop/mirror/contacts
https://developers.google.com/glass/develop/mirror/subscriptions
More Tips
Templates
JSP
http://freemarker.org
Scheduled jobs on server
CRON settings on GAE
Programming Google Glass - The Mirror API
By Eric Redmond
http://pragprog.com/book/erpgg/programming-google-glass
Quick Start Samples https://developers.google.com/glass/develop/mirror/quickstart/in
dex
Sample Code Online
Mirror-QuickStartSample-Java-GAE
https://glass-mirror-java-sample.appspot.com
Mirror0-HelloGlass
https://hello-glass-mirror.appspot.com
Mirror1-HelloWorld
https://hello-world-mirror.appspot.com
More Information
Website
https://developers.google.com/glass
http://arforglass.org
http://www.hitlabnz.org
Gun Lee
Mark Billinghurst