Web services that output Xml and Json as well as Xhtml?
You asked for it, now you’ve got it!
The Student Web Service released to production yesterday and the Financial Web Service went out this morning.
A big thanks to everyone who tested in eval. We can’t wait to see the cool things you build with this!
In our efforts to convert our XHTML representations over to XML and JSON formats, many questions have arisen along the way. Here are some of the issues we’ve wrestled with recently.
To make XML representations easier to parse, we decided to use the same namespace across an entire web service. For example, Person Web Service (identity) will have a uniform namespace across all of its resources:
We originally planned on a unique namespace for each resource type, but a single namespace makes XML parsing – and generation – much simpler. Note that the namespace doesn’t include the version number. The reason for this decision is one of simplicity – a client shouldn’t need to change their namespace when they want to consume a new version of the service, and the client already knows what version of a resource they are viewing because they have requested it in the URI. There was also a little uncertainty over whether we were creating a namespace ambiguity in our use of <SearchResults> as the top level element for all of our search resources. As in the case of version number, the client developer already understands the type of resource they are searching for by specifying it in their URI, so they can also know that within <SearchResults> is a list of the thing being searched for.
One of the benefits of the XHTML representation was that it made it easy to determine all of a search resource’s search parameters. Each parameter appeared as an <input> element in the search form, and along with it some possible values. From that a query string could quickly be derived. In the XML/JSON representation, all of the search fields are still listed, but not in such a way that they can be turned into a query string. The only guide as to what a search resource expects as parameters is contained in a URI, without much context or indication of possible values. For example, the current snapshot of PWS offers the following URI for its person search representation:
/identity/v1/person.xml?uwregid=&uwnetid=&employee_id=&student_number=&student_system_key= &development_id=®istered_surname=®istered_first_middle_name=&edupersonaffiliation_student= &edupersonaffiliation_staff=&edupersonaffiliation_faculty=&edupersonaffiliation_employee= &edupersonaffiliation_member=&edupersonaffiliation_alum=&edupersonaffiliation_affiliate= &page_size=10&page_start=1
It’s not exactly something we can expect a client developer to make sense of, especially since a URI is something that is designed to be opaque. This is something that can and should be overcome by sufficient documentation. For the search resource that provides the URI above, I would expect to see in its documentation a list of all of the possible parameters along with what the expected values to those parameters are.
A build of Student Web Service with XML/JSON/XHTML payloads for all resources (with the exception of GradeRoster) is ready for testing. This is in response to many of you who shared with us your thoughts via Uservoice, emails, chat, and/or face to face meetings. We understand that to help advance developer adoption of web services it is important we offer XML and JSON representations of our web services.
Things you should know:
- Production release scheduled for the end of this month.
- All uris, including public, have remained unchanged for Xhtml. Please see below for examples on how to get XML and JSON representations.
- Although we have a good build ready for testing there are still some bugs and design considerations we will likely fix before we release to production.
- If you already have access to the web service then no new authorizations are required from the Office of the Registrar.
- We plan on decrementing Xhtml some time in the future and will provide ample communication prior to that change.
SWS EVAL Test Environment examples:
1. Using Course Search to find all Math courses using JSON or XML. Append .json or .xml to the end of the search resource name.
2. Get public course information for Math 100. Append .json or .xml to end of the uri.
Developer documentation for these new formats is available here. The target date is about 2 weeks from now so please help us find bugs before ship date.
Your timely feedback and bug notifications will be greatly appreciated to us and the rest of the UW. Please send all issues and bugs to firstname.lastname@example.org.
Blogger William Vambenepe takes the stand that Amazon proves that REST doesn’t matter for Cloud APIs. He states that he doesn’t mean it’s a poor choice, but that other reasonable alternatives (e.g. RPC) are just as good. Regarding AWS, he asks:
Has this lack of REStfulness stopped anyone from using it? Has it limited the scale of systems deployed on AWS? Does it limit the flexibility of the Cloud offering and somehow force people to consume more resources than they need? Has it made the Amazon Cloud less secure? Has it restricted the scope of platforms and languages from which the API can be invoked? Does it require more experienced engineers than competing solutions?
What: Our first of three quarterly Ignite-style presentations from UW people using Web services –- 5 minutes and 20 slides each! Each presentation will be followed by a 5 minute Q&A period. We also have a slightly longer session scheduled on the Family Educational Rights and Privacy Act (FERPA), a topic of importance to anyone who deals with student data.
When: Thursday, December 2, 2010 from 3:30-5pm.
Where: Odegaard Undergraduate Library, Room 220
Who: Here’s the agenda…
• Virjean Edwards – FERPA and You
• Dan Boren – Web App Security: Two Things You Gotta Know
• James Renfro – RESTful Workflow
• Josh Ritter – Google Sites Integration via RESTful APIs
• Paul Schurr – The Genesis of XML and JSON Web Services
• Heather Sherman – Reporting with Google Visualization
• Jody Tate – Web Service Views Using jQuery UI
———————————– Web services discussion group info ———————————–
We meet once per quarter, from 3:30-5:00 PM in Odegaard Undergraduate Library, room 220. The meeting dates are: 12.2.10, 2.17.11 and 5.19.11.
To discuss Web services of all flavors, please email questions, suggestions, challenges, etc. to: email@example.com
Our Web service blog is located at: http://depts.washington.edu/ontheroa/
The UW Web services Registry is located at: http://webservices.washington.edu/
And finally, don’t forget a free service available to the UW developer community–Web Service Design Reviews. If you are designing, developing, or integrating a Web service and would like some help, call on us!
More information is available here: https://sig.washington.edu/itsigs/Web_Services_Design_Review
Last Wednesday we released a new update to SWS which included a new TestScore resource. You can find the developer doc here. The most exciting part of this release is that the resource comes in XML, JSON, and XHTML formats. Early next year we will be converting all our resources so that you consume all our Student Web Service thru XML, JSON, and XHTML.
Thank you to all those who helped us with the design and testing of this new resource.
- UW Admissions
- College of Education
- Computer Science and Engineering
- Health Services
- Physics Department
- Catalyst Tools Team
- Student Information Services
- Office the Registrar
Keep in mind we will be decrementing XHTML in future (major) versions of the web service but that time frame has NOT YET been determined.
REST dominance as an API is growing. Yaaaay! We’re doing something right.
The UWIT Application Integration Services team in partnership with our Student Information Systems team has begun the design and development of SWS Test Scores. This new resource to be scheduled for release by end of November will allow developers to programmatically access official test scores such as GRE, SAT, GMAT, TOEFL, etc… The primary motivation for the development of this resource is to support the UWSDB transition project, currently in progress, so we can move applications from UWSDB to Student Web Services. The second reason is to support an enterprise need to provide programmatic access to test scores without the need for a local data extract.
While we are in development you can learn more by referencing the SWS test scores RESTFul web service design doc and shortly after the resource is released we will provide developer docs.
Many thanks to the following departments for helping us understand requirements and create an initial service design:
- UW Admissions
- College of Education
- Computer Science and Engineering
- Health Services
- Physics Department
Please give us any feedback on the design as the next two weeks is when we will be best positioned to accept major design change requests.
An XML parser written in assembly:
It claims to be an order of magnitude faster when parsing XML and only 20Mb in size. An interesting point they make is the XML is not the technology to use when speed is needed.
9/17 – 9/24
Web Service Release
We plan on releasing either FWS and/or SWS with these new formats soon (within a couple months). PWS, IdCard, and DecisionSupport release time frames that includes XML/JSON has yet to be determined. The additional work required to complete the conversion is outlined below.
Visual Studio 2010 Conversion Issues
We were able to convert all our web services to VS 2010, .NET 4, and EntLib5.0. With that said, this took more time than any of us on the dev team anticipated. We found very unusual behavior in VS 2010′s checkin capabilities that confused developers on what files were actually being checked out for use and checked in for saving. For example on more than one occasion we found that VS2010 was automatically checking in our bin folder and all dlls contained within them without explicit knowledge from the developer. We also found checkins and outs to TFS (Team Foundation Server) to be horribly slow.
We found a .NET4 bug related to the XslCompliedStylesheet class that took some time to debug and plan on submitting a bug report to Microsoft.
We also found that our core framework when complied in Release mode did not inject (using Unity) our LogManager objects consistently. We still do not have solution to this problem however are currently looking into it. Right now all projects are being run with a Debug dll.
The last and most concerning problem related to this technology renewal upgrade is that we found the IIS7 w3wp.exe process pegging the CPU intermittently when our services were executed. We have yet to find a repeatable test case to help us debug this problem however have confirmed that all developers working on our sprint have experienced the problem on their workstations. We plan to test more on our test servers to isolate whether this problem is a Visual Studio 2010 desktop issue or a general issue that will show itself on our servers as well.
Good news is that we did not find any degradation in performance by moving to .NET 4.0 and Entlib 5.0.