Beanworks

Iterative Development:  Done Simply

Agile, Scrum (Agile case study)

Problem:  too much to do (geez, even with a staff like that?); priorities, requirements change frequently, emergencies happen

IT blackbox:  those on the outside don’t know what’s going on inside

Agile: response to waterfall method of software development:  values working software, customer collaboration, change, interaction

Scrum: agile methodology

Roles:  product owner, scrum master (keeps things going), team

Artifacts: product backlog, sprint backlog

(visual of scrum sprint.)

Daily scrum (c. 15 min. @ start of day) what’s going on, bring up to date

Sprint retrospective:  what went well, what to change for next time

At NCSU: used to tackle big problems in small pieces.  Using iterative development loosely based on scrum.  Just in time planning & documentation;  cross functional teams with an IT point person + developers; joint “owners”

NCSU toolbox:

Product & Sprint backlog: JIRA

Requirement confluence + JIRA

Planning: google docs + JIRA

Daily scrum

Sprint demo at product team meetings

6 week iteration: 1 week planning, 4 weeks development (realign as necessary) 1 week testing/release

Use 1 week to plan across multiple projects

1. high level overview of upcoming projects (3-6 mo)

Prioritize projects for next iteration

Add to google docs

2. Meet w/product owners for each prioritized project

3. At end of the week, re-prioritize based on estimates & time availability; back to googledocs (spreadsheet) w/ time estimates by individual.

4. Get it done: daily scrum, weekly review (how does progress look for cycle, requires work logging), subversion>JIRA integration

Issue burndown chart (should be going down as goal nears)

Test throughout cycle; demo at regular meetings, close tickets when closed, put comments in JIRA to document issues.

Challenges: multple small projects within a cycle: not traditional for agile practice

Lack of documented requirements (what are user stories and when do you need them; librarian teams work slowly); prioritization is difficult for library staff (work at release level), testing (how & when for small projects), no QA experts, simultaneously handle support & develop (juggling act)

Outcomes: positive movement across multiple projects.  Individual development efforts timeboxed, increased user satisfaction, increased flexibility to adapt (users hate seing no movement)  “Positive energy” feedback and librarians seeing movement

Resources: Agile Project Management with Scrum (books)

Vampires vs. werewolves: ending the war between developers and sysadmins

Each side’s goal is satisfying the villagers w/pichforks

Innovation is about risk, & you don’t take risks with people you don’t trust.

Reach out and build trust:

1. Test:  do you KNOW that your code works? does it work on the system; will it after system upgrade? (Uses Nagios to monitor uptime, what’s working, & what’s not; plug in sys upgrade & monitor from Nagios to see what to expect.)

2. Documentation: for the sysadmins, so they can know what your process/code is supposed to do, & who’s responsible, contact info, etc.

3. Puppet (sys config tool) book: Pulling Strings with Puppet. (uses ruby)

I am not your mother, write your test code

Ad hoc testing: where’s the safety net?

Automate testing:  regression testing (local code, local mods), make sure your code doesn’t break someone else’s, refactoring, reduce design complexity, specify expected outcomes

Hudson: continuous integration tool

Selenium: firefox plugin – does automatic searching of web site, makes assertions, asserts text in an xpath. (but xpath is brittle)

(working in ruby): rSpec, cucumber, rails testing framework; rSpec & cucumber primarily.  rSpec tests whole stack.  Cucumber has “features” with “scenarios” (tests instances)

Types of tests:

Unit tests; testing at function or method level.  Integration test would test interface between functions.  At high level, tests stack

What to test: assert call (assertEquals(…)); test branching – differing paths of logic; test during bug fixes (first write test to figure out the bug); test for exceptions, error-handling (try-catch);

Testing legacy code: start with bug fixing: high level, test the section (trying to understand what’s going on).

Run tests: isolate what you’re testing, & for dependencies on other processes (e.g., network, db, solr, external svcs)

All modern languages have unit testing frameworks

Result: fewer bugs, better design, freedom to refactor

Media, Blacklight & Viewers like You (WGBH Interactive)

Using fedora, lighttpd, solr, mysql, blicklight (media screen), jquery, PBCore (mirrors DC in a lot of ways) Question to libraries: http://tinyurl.com/c4l-pbcore – how can PBS make pbcore more useful to libraries?

Metadata apis: rdf/xml, oai pmh, unapi, embed uriplay (common way of embedding stuff across sites)

Fedora has problems with large data strings.

Left Quicktime rsp streaming -> h.264 pseudo-streaming using lighttpd

Apache is in front of fedora and streams data directly out of the data store.  Fedora manages the files, but doesn’t have to deliver them

Rights:  ODRL (metadata records are public, limited backend security policies).

Media workflows: have to integrate with whole bunch of asset management systems; no standard identifier control, so there’s extensive manual processing

Becoming Truly Innovative: Migrating from Millennium to Koha

New York University Health Sciences libraries, was on Millennium III.  Outside the library, IT departments in the medical center are merging.  HIPAA issues involved; circuitous route around firewall to medpac (OPAC).

When network security (IT) consolidated, policies made things break.  When intermediate server was disconnected, the ILS went down.

NETSecurity required all vendors to connect through Juniper VPN(etc)

Enter concept of open source: tested desktop level first with basic data on circulation, holds, acquisitions, serials, bibliographic & authority records

Circulation stuff: take from Millenium, normalize, put in excel, compare to other lists, enter into Koha

Bib/authority: millennium tags to marc fields not 1-1, not comprehensive, multiple subfields.  Migration tool did not work.  Used Xrecords: pulled from III (supply list of records in one line), put in XML, to XSLT, to marcXML, to Koha

XSLT needs to be customized for each institution; III codes didn’t map directly to koha codes (used conditionals, patterns)

Holds had to be handled manually

Migration took place from Aug 6 (freeze cataloging) to Aug 30 migration, aug 31 live. (contract w/III ended on aug 31)

Bibliographic records =80k, authority records = 5k, patron recordss = 11k; acquisitions = end of fiscal year – start over; serials was a problem

Outcome: good, o.k. – everything got in & done & everything is working fine; code is available for others, but you have to write your own XSLT

Caveats: mfhd  – no support yet, diacritics, cross linked items, multiple barcodes/call number, keeping track of old record numbers; pull patron records on day of migration, not before.