Decorative Robotic Plastering

A Case Study of Real-Time Human Machine-Collaboration in High-SkillDomains

Joshua D. Bard1, David Blackwood2, Nidhi Sekhar3, Brian Smith41,2,3,4Carnegie Mellon University1jdbard@cmu.edu 2dablackwood@cmu.edu3,4{nsekhar|briansmi}@andrew.cmu.edu

This paper explores hybrid digital / physical workflows in the building trades, ahigh-skill domain where human dexterity and craft can be augmented by theprecision and repeatability of digital design and fabrication tools. In particularthe paper highlights a project where historic techniques of decorative plasteringare extended through live motion capture of a drawing implement, informationrich visualization projected in the space of fabrication, and custom robotictooling to generate free-form running moulds. This workflow allows designersand craftspeople to quickly explore patterns through free-hand sketch, test ideaswith shaded previews, and seamlessly produce physical parts using roboticcollaborators.

Keywords:Motion Capture, Robotic Fabrication, Haptic Interface, Hybrid Skill,Human-Machine Collaboration

MOTIVATIONThe human body fosters a wealth of tacit knowledgevital to cultural, political, and economic dimensionsof human life. Think of the learned dexterity of a sur-geon's fingers, the buoyancy of a dancer poised toleap, or the deftness of experienced hands guiding achisel through natural wood. Despite the body's cen-trality tomany importantmodes of human endeavor,technologyhasoften replacedbodily skillwithmech-anized production. In the architectural arena, in-dustrial design and manufacturing tools have sig-nificantly altered the relationship of human craft tothe design and production of the built environment.The expressive skill of the craftsperson and designerare at risk when we consider the innumerable hours

of design creativity forced to flow through an archi-tectural intern's mouse finger or the mind-numbingrepetitiveness of supervising most industrial manu-facturing equipment.

HUMAN ROBOT COLLABORATION INHIGH-SKILL DOMAINSDespite the prolonged history of displaced humanskill in relation to industrial production, this neednot remain the case. Contemporary developments inrobotic fabrication and real time sensing are disturb-ing the equilibrium of the design and industrial man-ufacture of architectural building components. Thispaper outlines the development of digital tool-sets

Fabrication - Applied - Volume 2 - eCAADe 33 | 383

to enable collaboration between humans and robotsin the high-skill domains of the building trades. Anintuitive interface using live motion tracking (MO-CAP), digital projection, and gesture recognition wascreated to move designers and craftspeople awayfrom offline programming at a dedicated computerto using gestures and sensor-embedded smart toolsto interact with robot collaborators. This entailedconnecting the disparate worlds of constraint-basedmotion planning-which excels at open-ended, inde-terminate tasks, and real-time decision making-andCAD-generated offline programming, favored by de-signers andknown for robust geometric constructionand visual feedback during the design process.

RUNNING MOULDS, A CASE STUDY IN AR-CHITECTURAL PLASTERThis section describes the traditional craft of plasterrunning moulds and the relevant constraints consid-ered in robotically augmenting the process in con-temporary design practice. The running mould is anancient plastering technique used to construct or-namental architectural elements in interior and ex-terior applications (e.g., cornice moulding) (Van DenBranden and Hartsell 1985). The technique involvescutting a decorative profile into a piece of sheetmetal and running the profile repeatedly over mul-tiple guages of plaster as it cures from liquid to solid.Although running moulds are often visually com-bined with other plastering elements, the fabrica-tion process is distinct from plastering techniquesthat require casting into dedicated molds (e.g., den-til moulding). Running moulds can be prefabricatedin a tradesperson's shop or run in-situ on a construc-tion site. The technique requires an experienced setof hands for appropriate setup and execution dur-ing plaster's rapid curing process. The key constraintis to run the profile across the curing plaster withas close to identical passes as possible. Because re-peatability is essential to a smooth finish, the metalprofile is mounted to a sled, which is typically guidedby a set of rails or a fixed pivot. Human dexter-ity is required to smoothly move the profile across

the curing plaster, but the overall path of the profileand the pressure shaping the plaster finish are con-trolled by the guides. Being constrained by physi-cal guides when running profiles by hand, dictates afairly limited set of possible running mould geome-tries (e.g., straight lines, circles, and ellipses). Devia-tion from these shapes requires costly and time con-suming setup of custom guides.

Figure 1Morphfaux,decorative roboticplastering on freeform surfaces

Custom profiling tools were developed for a six axisindustrial robot to test new possibilities for the run-ning mould technique (Bard et al. 2013). Becauseall offline robot motion planning is reliably repeat-able, and six axis robots have a high degree of kine-matic freedom in space, physical guides became un-ecissary. This entailed the ability to explore free-formmoulding in complex three dimensional configura-tions, producing decorative profiles that would bedifficult, if not impossible to construct by hand (fig-ure 1). The workflow outlined in this paper respondsto the possibilities afforded by contemporary tech-nology to augment traditional building trades. If thelogical layout technique for traditional moulding de-sign required ruled drawing, using straightedge andcompass, then the kinematic freedom afforded byrobotic technology suggests a more direct connec-tion to freehand sketching. The following sectionsillustrate how bodily skill can remain central to de-

384 | eCAADe 33 - Fabrication - Applied - Volume 2

signers and tradespeople when exploring the aug-mented space of robotic construction through seam-less tangible interface in fabrication settings.

AUGMENTED FABRICATION CONTEXTSAs computation becomes more ubiquitous and em-bedded in our physical environments, many high-skill domains have become hybrid in nature, lever-aging the precision and repeatability of computa-tional tools along with the dexterity and expressiv-ity of human skill. Whether competing at Chess orconducting surgery in the operating room, hybridworkflows are often more effective than tasks per-formed by humans or computers acting on their own(Brynjolfsson and McAfee 2014). The building in-dustry has been late adopters of robotic technolo-gies. Although, the building industry has borrowedheavily from the manufacturing sector encouragingmany industrialized buildingmaterials and construc-tion methods, the primarily autonomous robotic as-sembly approaches found in many factories are of-ten ill-suited to the challenges of on-site constructionand the degree of customization in architectural pro-duction (Lee et al. 2014). The factory favors automa-tion, which requires the reduction of complex tasksto their lowest common and repeatable denomina-tor. The shop and construction site, however, favorthe efficient and varied application of dexterous skillto open ended tasks. The logistics and efficiencies ofthe factory floor cannot simply be re-created in theshop or on the construction site.

Advances in contemporary technology suggestpromising possibilities for human-machine collabo-ration in the building trades. Three types of technol-ogy central to these advances in fabrication settingswill be demonstrated in the case study of decorativerobotic plastering.

1. Real-Time Sensing: Human machine collabo-ration in fabrication settings relies on the ex-change of information based on actions inthe physical world. Capturing human gesturefor skill-transfer, tracking tools precisely in 3Dspace, and sensing material behavior in re-

sponse to CNC tooling are all made possiblethrough real time sensing.

2. Information Rich Projection Mapped to Phys-ical Context: The typical shop is littered withdevices used to bring a sense of measure tothe physical context of fabrication. Tapemea-sures, fixtures, jigs, clamps, and templatesallow material and tools to be used withintention. Information has been tradition-ally passed from paper drawing sets throughthese physical mechanisms to produce reli-able results. The advent of projection map-ping allows for a seamless transfer of infor-mation from digital models into the physicalspace of fabrication. Measuring grids, customlayouts, part labeling, and assembly informa-tion can all be project onto the three dimen-sional surfaces of the shop in real-time.

3. Seamless Incorporation of custom RoboticTools: Industrial robots encourage customtools andworkflows for shapingawide varietyof materials. Unlike dedicated machine tools,robots offer more flexibility when adapting tothe diverse demands often placed on archi-tectural fabricators and construction crews bythe custom requirements of specific projects.Robots become part of a flexible infrastruc-ture that can support customization and in-vention.

WORKCELL SETUPThe following section describes the physical work-cell developed around a plastering table for produc-ing running moulds and the software developed tocombine real-time sensing, projection mapping, androbot motion control in a hybrid fabrication setting(figure 2).

A six camera motion capture array is mountedabove a 1.2m x 2.4mwork table. Real time tracking ofa stylus allowsusers toexplore the constraint spaceofdecorative plaster moulding patterns through free-hand sketching on the table top. A custom streamingcomponent in grasshopper, developed by a collabo-

Fabrication - Applied - Volume 2 - eCAADe 33 | 385

rative team for aworkshop at the 2014 Robotics in Ar-chitecture conference, brings hand sketching infor-mation directly into the CAD environment (Schwartzet al. 2014). A custom calibration script automati-cally aligns all tracking with the lab robot's base co-ordinate system. Once calibrated other tools or workobjects can be tracked relative to the robot's coordi-nate system (figure 3).

Figure 2Physical work cellwith MOCAPcameras, ceilingmounted projector,and industrialrobot.

Figure 3Real time trackingof stylus withprojection of drawncurves on worksurface.

Visual feedback is projected onto the work surfaceusing a ceiling mounted, wide-throw projector. Pro-jection mapping is calibrated with a custom Pythonscript in grasshopper using hand-placed MOCAPmarkers. From the CAD environment design and as-sembly information can be projected onto the work-table including, shaded previews of potential pro-files, start and end-point locations of neighboringpieces in a pattern, and underlying grids formeasure.An interactive dashboard is also projected onto thetabletop and allows users to explore different pro-file curves and negotiate the part to whole relation-ship of individual tiles in the overall pattern (figure4). The user can move fluidly between design itera-tion with the stylus and parametric pattern genera-tion displayed through the projector.

As the pattern develops through iteration, theuser can generate drive curves for robotic plaster-ing on the work surface directly from the stylus seedinput. Users can choose to smooth the raw inputto remove or amplify noise from the hand-drawnmark. Robot motion control is also handled withinGrasshopper using the HAL plugin (Schwartz 2015).Custom Grasshopper scripts help with motion plan-ning in order to avoid joint errors from the input ofhand-drawn guide curves. A custom end of arm toolallows for plaster profiles to be quickly interchanged(figure 5).

DEMONSTRATIONA sample pattern was tested to demonstrate the po-tential of augmented fabrication workflows in archi-tectural plaster. The authors developed a parametricpatterning script that generated radial arrays arounda group of asymmetric obstacles commonly foundin a contemporary ceiling plane (e.g., light fixtures,columns, HVAC, fire suppression). Historically plas-ter rosetteswere installed aroundchandeliers in care-fully symmetric patterns. Users seed the algorith-mic pattern generation with two curves, sketchedusing the tracked stylus. An inner and outer draw-ing boundary and a measuring grid are projected atone to one scale on the work table to aid in hand-

386 | eCAADe 33 - Fabrication - Applied - Volume 2

Figure 4From right to left:projection ofdashboard on worksurface, projectionof fabricationinformation prior todrawing, projectionof shaded preview,final plaster samplefrom drawing.

sketching. Once the two seed curves are generatedusers can see a shaded display of the full plaster pro-file projected around the ceiling obstacle. Users canexplore the parameters of the pattern, changing thedensity and spacing of each input curve. Users canalso iterate the input curves with further sketching,getting visual feedback in real time (figure 6).

Figure 5Custom end of armplastering tool withinterchangeableprofiles.

In this exampleworkflowthevirtual statusof rep-resentation, which typically precedes constructionand is constrained to screen or paper space, perme-ates the physical space of construction. The collapsebetweencontexts for representation andcontexts formaking allows for haptic exploration of design con-straints that are both materially pragmatic and algo-rithmically extensive (figure 7). Rather than replacinghuman skill through automation, human gesture ex-presses itself at the intersection of physical and digi-tal design processes, augmenting what is possible inthe high-skill domains of the building trades.

CONCLUSIONThere are many areas where hybrid fabrication envi-ronments can augment the building trades. Futuredevelopments of this case study will include devel-oping a Kinect based gesture interface to seamlesslynavigate dashboard settings while carrying out fab-rication tasks in a shop environment. We would alsolike to develop a robot mounted projector where in-formation can be mapped to any surface within therobot's work cell. Posing the robot for specific pro-jections would allow for a larger and more flexibilityprojection environment. Lastly, the team would liketo develop an automated material delivery mecha-nism, integrated with the end of arm plastering tool.Currently plaster is distributed by hand, based on vi-sual prompts from the projector. Automated deliv-ery could be more efficient and require less materialwaste in the initial passes.

Fabrication - Applied - Volume 2 - eCAADe 33 | 387

Figure 6Pattern generationand explorationusing handsketching andparametric design.

Figure 7Sample patternplastered fromhand drawn input.

Emerging technologies in robotic fabrication andreal-time sensing enable small-scale production, af-ford a higher degree of customization and allow de-signers greater access to the means of building pro-duction. The difficult relationship between the hu-man body and industrial machines can be reformu-lated in this new context. Ultimately a collaborativerelationship can emerge where the salient charac-teristics of human skill and machine precision workin tandem toward augmented paradigms of fabrica-tion. Working toward robot-human collaboration inhigh-skill domains entails defending the sustained,often pleasurable, mind- and body-work required forarchitectural design and production.

REFERENCESBard, J, Mankouche, S and Schulte, M 2013 'Morphfaux',

Robotic Fabrication inArchitecture, Art andDesign, Vi-enna, pp. 139-142

Van Den Branden, F and Hartsell, F 1984, Plastering Skills,American Technical Pub., Alsip, Ill

Brynjolfsson, E and McAfee, A 2014, The SecondMachineAge, Norton, New York

Lee, S, Lee, K, Kim, J and Han, C 2006, 'Human-robot co-operation control for installing heavy constructionmaterials.', Autonomous Robot, 22(3), pp. 305-319

Schwartz, T, Bard, J, Gannon, M, Jacobson-Weaver, Z, Jef-fers, M and Tursky, R 2014 'All Bent Out...', RoboticFabrication in Architecture, Art and Design, Ann Ar-bor, pp. 305-317

[1] http://hal.thibaultschwartz.com/

388 | eCAADe 33 - Fabrication - Applied - Volume 2