University of Washington Emerging Technology

Presentation

Presented by John Musser (Founder of Programmable Web and a great guy to talk with.)

John provided a well rounded presentation on enterprise mashups. Cleverly debunking the myths and focusing on the data and realities of enterprise mashups. He also gives an analysis of mashup tools and expected enterprise mashup future trends.

The Hype:

ZapThink’s 2008 SOA Forecast:

“The world of enterprise mashups will come into its own in 2008, and become what many people
are calling the “killer use-case” for SOA.”

The Surveys:

Source: Information Week, September 2007

I like this public access pie chart where it shows 43% of IT web applications are either developing or have deployed applications using data from publicly accessible web sites. I wonder if that meant both web pages and web services? With the trending towards RESTful APIs returning payloads that are XHTML and ATOM based, does making the difference between web pages and web services matter any longer?

What is a mashup?

Combines multiple web services into something new

Addresses a specific need

Developed quickly by an individual or small team

Rich user experience (optional)

I think these points are excellent in that our vision for ROA, which used to be SOA for the UW, hits upon our goal of serendipity and quick creation of innovative web applications. This is in contrast to our past designs where we focused mainly on long-living applications.

Here are some points that make looking at mashups different than the way we currently or used to see things.

Lightweight application combining data and services from multiple sources

Developed inside an enterprise

By either IT or business staff

Created in days not months

Uses a Web Oriented Architecture(WOA)

For the UW this would be “uses a Resource Oriented Architecture (ROA)”

Often uses internal + external web services

Done at data, logic and/or presentation layers

What makes mashups different?

See slide #10

Adapted from ProgrammableWeb.com Web2.0 Expo slides

What I like best about this matrix is that it gets to the heart of the same decisions and tradeoffs we made here at the UW; both to our webservices and application development strategy.

Web 2.0 is definitely coming into the enterprise and here is a list of those things:

Open APIs

SaaS: Software as a Service

RIA (Rich Internet Applications)

Dynamic scripting languages

Cloud computing

Widgets

Open Standards

The Long Tail of IT - a really good slide on how mashups address the long tail of IT which is characterized as:

Unmet business demand for IT

Ad-hoc, tactical solutions

Manual processes

See slide #19

I like this pie chart from Information Week, Sept 2007 which asks the question about how folks feel about non-IT staff developing their own mashups. Which for the record I believe is the right thing for the UW to do! Of course making sure we work through all the important challenges while we advance to that end state.

I like the comparison that Microsoft Excel is currently still the mashup tool of choice. While this new Web 2.0 era is more about web based accessibility of mashups as well as real time access to that information.

John also makes another comparison of mashups that makes sense. Enterprise mashups are focused on implementations that target a specific business problem. Making solutions much more agile vs our current monolithic systems which were created to meet global central processing. Now those systems are expected to solve targeted and quick business problems which it can’t. This of course results in business opportunity loss and frustration directed at many IT shops.

Web Oriented Architecture

John presents a new acronym, WOA, which he describes as Web Oriented Architecture however his description fits almost perfectly with Resource Oriented Architecture. Especially with the comment that WOA is a unified means to identify resources online.ROA is what we currently subscribe to here at the UW.

See slide #29 for data related to the percentage of open APIs by type. RESTful APIs rise to the top with 68% seen in the wild by ProgrammableWeb.

How will you build your mashup? Fundamental commonality: HTTP, REST, XML, RSS, Atom, Ajax, XHTML

List of mashup tools presented:

SnapLogic

Kapow Technologies

IBM Lotus Mashups

IBM Infosphere Mashup Hub

IBM WebSphere sMash

Top 4 enterprise mashup challenges:

1. Immature Marketplace - early adopter phase with lots of change
2. SLAs for APIs - lack of SLA as a barrier to adoption
3. Security - data access rights; lack of identity standards; compliance regulations
4. Data quality and trust - applies to both internal and external data

Mashup Advice for IT
1. Beware the hype, but don’t ignore
More opportunity than risk here, and it’s going to happen anyway…

2. Got SOA? Make it a mashup platform
Mashup-enable IT infrastructure: use open standards, expose services

3. Start simply
Small apps, pilots, evaluations

4. Think tools, both for IT and business
Tools to enable wider adoption, speed creation, enforce policy

5. Add governance as needed
Mashup-aware policies for security and external services

Trends to watch:

Open web technologies driving enterprise mashups

SOA + ROA (sorry I just could not write WOA here)

Rapid growth of enterprise mashups APIs and tools

SLAs beginning to appear:

Google Maps Premier (starts at $10K/yr)

Amazon’s new SLAs (Silver $100/month; Gold $400/month)

Impact assessment of Mashups in The Enterprise

eTech key take aways:

Our UW ROA strategy align strongly with the mashup environment that John has described for us.

We have focused much of our energy on building out core webservices; now we need to spend equal time understanding how to deliver a mashup infrastructure and build agile web applications that allow our UW businesses to meet business demand in real time.

I recently had the privilege of teaching a class on web services for INFO 344 Winter QTR. I had a blast and I hope the students had fun too. Web services is something I am intimately engaged with here at the UW so I thought it was an appropriate subject for me to lecture on.

During my lecture I had the students participate in an activity that allowed them to share their ideas on how new web services on campus would benefit our university. The class was split into 5 teams and I have listed each team’s idea below (close to verbatim). The ideas were quite creative and my hope is that some of you who own some of these system will find these posts interesting and insightful. You can aggregate the thoughts of these posts to glean the general theme of what our students (40 of them) in INFO 344 would like to see.

Catalyst Doc Share

Use Case:

Students collaborating on essay

Client App:

Browser UI with basic word processing capabilities integrated with Microsoft Office and Open Office software. A required browser plug in would have to be installed to update the documents using client software. The plug in allows the office clients to update a doc (creating incremental versions) every 15-30 seconds. (Word processing “chat” program)

Web Service:

Keeps track of document versions both from browsers and client apps (MS Office and Open Office). Incremental versioning

Later expansion to Excel and Powerpoint docs

Class Registration visual schedule

Use Case:

Class registration visual schedule creation,integration & filtering based on time and requirements. The current way of manually entering in and adjusting schedules on a calendar is time intensive and prone to mistakes.

Client App:

Drag and drop course information to visual calendar using same browser window

Web Service:

Course catalog and schedule finder/search

FTP Backup App

Use Case:

Allow students to easily backup their files from their desktop to students.washington.edu

Client App:

Auto-save mechanism which prompts users to save files during desktop log off. Can be saved to students.washington.edu or deskmail based on user choice. This can be done thru FTP or SMTP transfer.

Web Service:

“We’re not trying to reinvent the wheel” using UW resources for this

Student registration and course planning

Use Case:

Student uses registration system and does course planning

Client App:

New schedule finder that combines maps, visual schedule, and route finder. The registration system will have a brand new UI. Plug in to allow for up to 5 SLNs which does:

1. Based on interested courses the system does a comparison of schedule time slots - looks for free/busy times
2. Output of possible schedules (best bets)
3. Sorted by options: distance, compact (Shortest) time schedule

Display visual schedule and improved maps

Web Services:

Google Maps API

For the other web services, UW already has most systems in place already for this to happen

Improved class calendar

Use Case:

Allow students within MyUW to easily see their class schedules that is enhanced so they can see all upcoming assignments, labs, tasks, in one place.

Client App:

Within MyUW pull up my schedule in a way to easily see what I need to do next for my class based on my syllabus. In a way similar to project management tools.

Web Service: Course catalog and course assignments service

I recently provided to the web services pilot team a short paper describing the value of the ATOM syndication format as a general-use web service format for an XML payload. ATOM is most well known for its use in web feeds or web syndication. In addition to syndication, we have been finding many uses of ATOM related to web services, specifically with REST-based web services that provide periodic updates or publications. The most prevalent example of ATOM- and REST-based web services is the Google Apps support for Gdata which is based on ATOM. Google provides web resources using the ATOM syndication format and ultilizes the APP (Atom Publishing Protocol) for creating and updating Google resources such as calendar, email, and blogs.

So why the fascination with ATOM, and does it have a legitimate use along side REST-based web services at the University of Washington?

https://dada.cs.washington.edu/mediawiki/index.php/WSPilot_ATOM_Research

Please let me know what you think.

After about 6 months, 30+ meetings, 1650 lines of code, 12 candy bars, 1 RESTful web services book, and 10 bags of chips; we completed our SOA pilot. Anything you want to know about our pilot can be learned here at the UW Computer Science and Engineering wiki https://dada.cs.washington.edu/mediawiki/index.php/Web_Services_Pilot which can only be viewed via UWNetID. Anyone outside of the UW who wants to learn more please contact me (ttchang@cac.washington.edu) or webservices-pilot@washington.edu.

The “PILOT” webservice is here (http://webservices.washington.edu/courses/) DISCLAIMER: This is an experimental service, and is not intended for production use.

Here is a quick take of our pilot:

Pilot Project Description

The following criteria were used to select the pilot project:

1. Demonstrates the benefits of a Web service interface for accessing central data stores;
2. Provides definite business value by removing costly redundancy between departmental systems;
3. Exposes data that are considered non-sensitive – preferably publicly available — to minimize the risks and complications of dealing with confidential or even sensitive data (e.g., anything protected by regulation, such as FERPA or HR data, or even anything of an un-regulated, but sensitive nature);
4. Does not create significant problems for consumers if unforeseen problems develop and the service needs to be re-engineered or reformulated in a major way.

After reviewing potential pilot projects against these criteria, the group chose to begin work on the UW Course Schedule Data Service. This service satisfies the above criteria as follows:

* It involves data that are already available;
* The data are not sensitive;
* The need for the data has wide enough appeal to serve the “demonstration” goals of the pilot;
* It is straightforward and simple enough to serve as a good vehicle for all parties to familiarize themselves with Web Services:
o For internal developers to learn to write services;
o For campus units to learn to write applications that consume them.

The Course Schedule Data (aka “Time Schedule” data) reside in the UWSDB. However, because of the sensitive nature of student data also contained in the UWSDB (regulated by FERPA) and the lack of a fine-grained access control mechanism, consumers of just the public course schedule data can’t easily be given access to UWSDB. A Web service interface to this data resource provides a convenient way to segregate the sensitive from the non-sensitive data. The Web service, therefore, provides a convenient mechanism for wide and unfettered access to non-sensitive (course schedule) data without risk of inadvertent release of sensitive data.

A HUGE thanks to Erik, Dan, and Scott from UW CSE as the driving force on this project and the code writers for the webservice!!! This was truly an example of great UW collaboration including the following organzations across campus:

1. CSE (Computer Science and Engineering)
2. Office of the Registrar
3. College of Education
4. OIM
5. Catalyst
6. C&C
7. Graduate School

I tend to think of services from the perspective of core business data or processes vs thinking about any specific applications which will require a service. Thinking only about the consuming applications themselves might cause organizations to create siloed services only applicable to that one or two consuming application. This model over time will yet again not resolve the redundancy issues across IT but potentially make it worse.

I guess a hybrid way of thinking is necessary. Think about a couple applications which require core data or processes to ensure there is a direct business need, then think about how the service should be developed to take into considerations “all” current and future applications, so as to not develop services to only meet the current demand.

So a common question which arises when faced with this dilemma is do we architect for the future or build for today? We want to do both. A service development team will be pressured to meet current business need and not have the ability to design services for tomorrow. Of course assuming it takes longer to develop services which addresses future demand as well as current demand.

Is it time to take a second look at the development paradigm for SOA? Is it necessary that one team be the sole development team for specific services? How about we change things a little and follow a “open source” development paradigm which any one (or any one approved by the domain/business owners of the service) can potentially contribute to the development of a webservice? Sure the domain/business experts should probably still perform the design of a web service however shouldn’t the development of the code be anyone who is willing to contribute? Lets take this scenario as an example:

1. An organization has determined a specific webservice is required to expose a set of core business data
2. The initial demand is driven by the HR department however everyone knows that the data will be needed in most parts of the organization over time
3. The HR dept has funded the development of the webservice to meet its specific business need
4. The central IT development team has been tasked to development this webservice.
5. HR has feed requirements to the central development team however the development team being SOA minded, notices that the requirements are specific to HR only and doesn’t address future enterprise needs across the organization. The development team sees a future need for this web service across the organization and proposes a different design which can meet HR requirements and future needs but will take 3 more months to finish.
6. The HR dept has a critical deadline which they cannot move out and says NO to the new design proposal.
7. The central IT group is forced to code to the original design request however also designs hooks into the webservice to allow for easy extensibility so other depts can modify for its own needs in the future.
8. After the web service has been created for HR on time, the Finance group has approached central IT for a similar webservice. The Finance group has noticed that the current webservice does not fully support their own needs due to a lack of additional data elements required for their business.
9. The central IT developers shows the finance IT development team the hooks to allow for extensibility. The finance IT development team specs out their business requirements and integrates themselves into the central IT development lifecycle. Using the same central code repository (source safe), bug tracking, change control, test processes, and other development tools and methodologies which allow finance IT to modify the current web service to meet their needs while ensuring other consuming IT applications do not break.
10. This is a joint partnership between central IT and other IT depts across the organization. The development and testing of the websevice is now jointly owned and managed. All IT depts and business owners who have a stake in this webservice drives the governance of code changes, feature requirements, bug triage, and other development matters.

Rather than being responsible for all development activities for their entire organization, central IT is now in the business of establishing a solid development infrastructure for the rest of the organization to develop upon and governance of IT is now shared across the organization (aka UW).

How about that for a change in development paradigm for UW or any organization?

We’ll be at the Web Services discussion group this afternoon talking about our recent SOA Workshop with the Burton Group (which I blogged here ).

3:00 pm in OUGL 220 - come on down!