Measure Twice, Cut Once

I’ve come across a quotation from management guru Peter Drucker many times. The quote is often given as follows, “You can’t manage what you don’t measure.” It’s usually given as evidence of the importance of developing, collecting and maintaining good data. Sometimes the quote is given as “what gets measured gets managed.” Which turns the quote on its side a bit, as when Paul Barnett gives it as part of a longer quote “what gets measured gets managed – even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.” In this context, it’s used as a warning, to be careful about what you are measuring because what you are measuring will draw your attention and influence your actions.

There is no evidence that Drucker said either of those things.

A similar quote is often credited to W. Edward Deming who said, “if you can’t measure it, you can’t manage it.” Again reflecting the importance of measurement for effective management. The only problem is that what he actually said was “It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth” – my emphasis. Either formulation of the quote, form either author, remind us to exercise caution when we are deciding what to measure.

These quotes, and their popularity, show the importance of and dangers in choosing what to measure and what to manage. On one hand, it is important to establish criteria for success. Criteria that are easily expressed in measurable attributes, like profit increases and cost savings in business or collection size and number of visitors in libraries, can capture our attention. They make it easy to communicate the story of the organization – before we had less, now we have more! But these criteria only tell part of the story. And the story they tell may be the wrong story. As the famous Einstein quotes says, “Not everything that counts can be counted, and not everything that can be counted counts.” (BTW, he apparently never said that.)

As libraries look for ways to measure their progress and tell their stories, they need to beware of a pitfall that psychologists call surrogation. Wikipedia describes surrogation as “a psychological phenomenon in which the measure(s) of a construct of interest evolve to replace the construct itself.” (https://en.wikipedia.org/wiki/Surrogation) Like,s ay using the size of your acquisitions budget to gauge the quality of your collection. They go on to give the example of “the tendency for managers to lose sight of the strategic construct(s) the [performance] measures are intended to represent, and subsequently act as though the measures are the constructs of interest.” It’s something to think about while you are focusing in our your ARL rankings or your IPEDS data.

The example on my mind these days as we are heading back to school is the propensity of universities to tout the quality of their incoming freshman. That’s great, you found great students and you provided an opportunity for them to succeed. But a great school would take any student and launch them on a stellar career and life. I suppose it depends on your “strategic construct.” If you want to educate students, then the quality of your incoming class is meaningless. If you have other goals, like moving up in the college rankings or cultivating donors, then maybe that is a good measure.

So, as Drucker might have wanted to say, what gets measured, gets managed. So be careful what you measure.

Advertisements

Thanks, but no thanks for the feedback

How well do you take feedback? Recently my wife accused my of not taking feedback well. Needless to say, I didn’t take this feedback well. I want to be good at receiving feedback, but I usually come up short of my best intentions. I also want to be good giving feedback too. On one hand, I have a lot of feedback for people – it’s amazing how much better the world would be if people listened to me. But people tend to be less than receptive. Often they will flat out tell me that I may have a good point, but the way I delivered the feedback makes it hard to hear me. It’s a very effective way to protect themselves from the feedback – and it makes me think twice about providing feedback in the future.

Why is giving and receiving so problematic? We all need feedback in order to do our jobs, to know if others are doing their jobs, even to know what our jobs are. Without appropriate feedback I’ve spent countless hours (really, I don’t want to count them), working on the wrong project, built tools that no one has ever used, and driven myself crazy guessing what my co-workers were up to. Without regular feedback at work, things can grind to a halt, misunderstandings may start to pile up, the quality of work often diminishes, and your satisfaction with your job can tank.

One reason why we don’t deal with feedback well is that it challenges our sense of ourselves. When we feel a threat to what we believe about ourselves, most of us will double-down in order to shore up our weakened facade. You, like me, might think that you’re open-minded. That when you are presented with facts that challenge your convictions, you are open to changing your mind. Maybe you are, at least some times. But research covered in books like Influence: the psychology of persuasion, Thinking Fast and Slow,  and others tell a different story. These books tells us that no matter how rational we may be in our moments of reflection, we will still rely on shortcuts to save time and energy.  We could take the feedback, process this new information, and make appropriate adjustments within our opinions. But it’s just easier to dismiss the feedback as unwarranted, unwanted, and beside the point. This works surprisingly well, most of the time.

But there are times when we need feedback. It makes us better workers, but more importantly it gives us the opportunity to be better people. In their book Thanks for the Feedback: the science and art of receiving feedback well (even when it is off base, unfair, poorly delivered, and frankly, you’re not in the mood), Doug Stone and Sheila Heen make the case for valuing the feedback that you are receiving . It’s worth reading if you get a chance. They also lay out some ways to get better at receiving feedback. I’ll offer just one: practice receiving feedback.

A few years back at an anti-bullying symposium where I worked, Fran Sepler said one of the best ways to counter bullying was to create an organization that values open feedback. In order to facilitate this, she suggested that people should ask for feedback often. After a meeting, turn to the person next you and ask how you did in the meeting. Ask for feedback on that  presentation you gave. Ask for feedback on any of your job duties. In addition to getting useful information about yourself, you’ll learn more about and your coworkers: what they value, what their concerns are, how they view you, and more.  When they provide feedback, say thank you – and then ask for more if appropriate (it’s important to respect people’s time).

Through practice, I hope to make myself more open to feedback. And maybe through my openness to feedback I can influence others let down their guard. Being a role model for receiving feedback will encourage people to share their feedback more frequently and hopefully to be more open to receiving feedback. By making feedback a normal part of our lives, it should lose its power to trigger our defenses, make us more comfortable with hearing feedback (both negative and positive), and help us to be the people that we want to be.

Fun with Google

There is a map in our ILL office.  ILL staff indicate libraries that we loan materials to by sticking a white pin in the map and they indicate libraries that lend to us by sticking a gold pin in the map.  If you are a map person, or even if you are just a library person, it interesting to check out places where things from our library have gone and it is interesting to see where things have come from.

I thought it would be interesting to see if I could duplicate the map on Google maps – and for the fun of it see if I could create a simple way to update the map.  The first thing I needed to do was get a handle on KML.  KML is a flavor of XML that Google uses to encode data about maps.  Once I had a handle on KML, I needed to find a way to get the information to populate the KML.

Like most university libraries, my library uses OCLC to facilitate lending between other libraries and us.  Additionally, the library uses ILLiad to manage that work.  Unfortunately, I don’t have direct access to the data in ILLiad, I’m not even 100% sure what data I could get out of it.  But I could get our ILL Librarian to get me a list of OCLC symbols.  And this is where things start to get interesting.

In order to get information about a library, I used the WorldCat Registry API.  From the registry I can get the name of the parent institution, name of the library, address of the library, the latitude and longitude of the library (very important for Google Maps), and a host of other information.  Pretty much everything I needed to populate my KML data.

So I wrote up a little php script that read through my list of OCLC symbols and searched the WorldCat Registry API for each one.  Fairly quickly I realized that many more symbols were failing than should.  Most often this was a result of a symbol  having more than one location to choose from.  So I tweaked my script to go through each location returned and make its best guess at which one was correct.  Which pretty much meant the first record that had a reasonably complete address.

Now I had a way to get the information about the libraries my library had been engaging in ILL with, but how was I going to get it into Google Maps.  At first I just crammed it directly into KML format.  In my hopeful first days the script generated live data.  However, as you might imagine, this resulted in considerable lag time and ultimately Google just ignored my data.  Next I output the KML data into a separate file.

Outputting the data was all well and good, but ultimately posed some challenges for updating the data.  I had always intended that the data would be able to be updated.  Entires needed to be added, deleted or changed.  I could just periodically run a new list of OCLC symbols through my script and overwrite the old file with new, updated data with any new entries and minus any that needed to be dropped.  However, I still had some failed WorldCat Registry searches that needed to be handled.  And not all of the ILL from my library goes to places that have an OCLC symbol.  So I was hoping to be able to manually enter data, when necessary,

My first solution was to update the KML file using the XML support in PHP.  While this was fun as a mental exercise, it was pretty obvious that I needed something a little more flexible and easy to deal with.  I went about moving my data into a MySQL database.  However, around this time I saw someone talking about Google’s Fusion Tables.  Fusion Tables seemed like a perfect fit for my project.  Fusion Tables are built to work with Google’s other products, like Google Maps.  Fusion Tables are smart enough to know when they are dealing with location data (most of the time) and you can import the data into a map without converting it to KML.

Ultimately, I didn’t go with Fusion Tables.  I wasn’t sure that I ultimately wanted to host this library data in my personal gmail account.  And I suspected that other people in the library would have some issues around managing the data this way.  So at first I stuck with my MySQL database.  But then my university contracted with Google for Gmail accounts with Google Docs and Sites and …  No Fusion Tables.  But I could access a Google Spreadsheet through the API.  And I could easily share the document with others in the library without having to ask for a personal email.  It was worth a try.

Ultimately, I ended up storing the data in a Google Docs spreadsheet.  I have a script that initially populated the data and two other scripts.  The first script lets to put in an OCLC symbol and have the data automatically added to spreadsheet.  This script handles the multiple WorldCat Registry entries with more subtlety, allowing the user to select the addess that they want to use, even providing a map as a visual aid.  The other script allows someone to manually enter an address and add it to the spreadsheet.  Again, with visual aids and the whole nine yards.

I should mention, that I found the latitude and longitude data in the WorldCat Registry to be less than reliable.  It was frequently absent.  Sometimes it was transposed, so the lat was the long and vice versa.  Sometime, I don’t know where it came from.  So I started using the Google Maps API to get latitude and longitude coordinates based on the address in the WorldCat Registry.

For the sake of my presentation here, I moved the data back into Fusion Tables.  Here’s how it looks:

It’s not clear that WP is playing nice with Fusion Tables right now, so I’m not sure that is going to show up, so here is the link – https://www.google.com/fusiontables/DataSource?snapid=S285421YrX2

Support the Home Team

Recently, William Denton posted an email on the NGC4Lib list about a talk he is giving at Access 2010.  In the description of the talk, he included the following:

Reference librarians are whiny and demanding.
Systems librarians are arrogant and rude.
Users are clueless and uninformed.

I started my library career as a reference librarian, but have spent the last 9 years working in systems.  During all that time, I have been an active library user.  So I guess that makes me whiny, demanding, arrogant, rude, clueless, and uninformed.  I’m sure I could find someone who would call me any of those things.  We all probably experience moments where we exhibit those unattractive traits.  It does little or no good to classify our colleagues and our users that way.  But response to the post makes it clear that those labels resonated with people.

So, why do we feel this way?  I’m sure there are many things that contribute to this feeling, but I want to focus on one issue, how our systems department interact with the rest of the library.  I currently work in a department called “Library Systems Support“.  This sense of support defines how the department interacts with other library staff.  While I don’t think about it from day to day, it still shows up in the interactions with staff members and in decisions that are made by management.  It makes the library staff view systems as people who work for them.  And it makes the systems department view the library staff as a drain on their resources.

 

Dilbert.com

I believe that working in partnership between systems and the rest of the library will help us break down the barriers that breed the hard feelings expressed above.  Of course, many library system departments do work in partnership with their colleagues and most or all probably do at times, but many libraries do still view their systems department as the “back office” operation which supports the “front office”, reference and special collections, operations.  And that dynamic fosters relations where not only is it likely that departments will have problems communicating, but it is often in the departments interest to not communicate.

When systems works as a partner with other departments in the library, both players can focus on the users and on the businesses processes of the library.  When systems serves as support for other departments and is not involved in the business of the library, the needs of the users and the definition of business processes becomes a negotiation.  The end result is that neither the user or the library is served.  Instead, the individual departments are served.  The things that the staff think are important are preserved.  The things that don’t directly impact the staff are the first to go.  Working as a partner changes the game.  Instead of starting from either department’s goals, both parties can work from a common goal.  Sure, that’s a bit idealistic, but the likelihood that focus is kept on the end result, rather than on who’s going to do the work, increase significantly.

It isn’t all milk and honey.  Working as a partner with other library departments is going to make more work for systems.  It will probably make more work for the other departments as well.  But working together will break down barriers between the departments.  So Systems will see Reference as more than whiny and demanding.  And Reference will see Systems as something more than arrogant and rude.  And we’ll realize that the users aren’t clueless and uniformed, they just need our help.

What is Librarianship

I recently came across a post on Rory Litwin’s blog about what “librarianship” is.  This is a topic that is of interest to me, in part because it is my chosen profession, but also because I think there is confusion among librarians about what librarianship is.  Librarianship, to me, is a profession.  This distinguishes it from being just a job.  When you are the member of a profession, it is your responsibility to think about what that means.  Just as an example, on one of the email lists that I subscribe there has been a lengthy discussion about privacy and the ALA Code of Ethics.  As a “professional”, I believe these are the kinds of things that you you should be thinking about.  When you have a job, you just show up and do what you are told.

So what is the confusion about librarianship?  I think Rory’s post hints at it, but I see it more pronounced on some of the email lists that I monitor.  People talk past each other on a topic because they don’t start out with the same assumptions.  To go back to Rory’s post for a minute, as someone points out in the comment, Rory talks about the fields of “web designers, information architects, web searchers, information scientists, user experience experts” belonging to other professionals, but then claims that librarians should acquire greater knowledge about “scholarly communities, the research into reading behavior, learning theory, media studies”, although those are fields that also belong to other groups.

I think the division created by Rory is informative, and relevant to the failed connections I was talking about earlier.  On the one hand is the a list of “technical” disciplines we should be wary of and on the other is a list of “pedagogical” disciplines that we should embrace.  The same dichotomy influences discussions on the mailing list mentioned above.  Arguments about what machines can/will do to make human intervention unnecessary versus how our users really use the machines can go on for weeks and usually only make both sides dig in more (Although, I do think recently it seems like there has been a little give on one or both sides).  The sad thing is that, for the most part, librarians aren’t experts on either the technical or the pedagogical issues.  We flounder around grabbing for whatever is popular (web scale!!!) or what catches our fancy (active learning!!!) without the training to back it up and without the research to show that it is worthwhile.

So, what is librarianship?  I can’t say that I know, but the idea I have been mulling around in my head since reading Rory’s post is that librarianship is about making sense out of the world of information.  Making sense in the broad sense in terms of defining the information universe, creating ways to describe the information in the universe, and developing ways to identify and access information.  And also, making sense in the micro sense, such as teaching information literacy to a class (or a single student), cataloging a book, building a local collection, or making a web site that helps users find what they are looking for.

To make this sense out of the incredible amounts of information out there and make it available to that person who walks in the front door or surfs over to your web site, you do need all the skills that Rory listed: an understanding of scholarly communities, the research into reading behavior, learning theory, media studies and you also need to be web designers, information architects, web searchers, information scientists, user experience experts.

Coping with Complexity

Last week I attended a presentation by Don Norman, the author of The Design of Everyday Things and many other excellent books.  His forthcoming book will be entitled Living with Complexity. Several points from Dr. Norman’s presentation got me thinking about how I could apply them in Libraryland.

The first point was that simplicity exists only in your head – the world is always complex.  The example used in the presentation was the Unix command prompt.  To the experienced Unix administrator this is a familiar and comfortable place.  A place where the experienced administrator can make magic happen.  For the user who has limited, or no, experience, it isn’t simple at all.  In fact it may send them to the local bookstore, or Amazon.com, looking for a book to tell them what to do next.  In contrast to the apparently simplicity of the Unix prompt, Dr. Norman showed a picture of the cockpit of an airplane.Cockpit of 747 The cockpit of the 747 in the image to the right seems very complex to most of us – at least those of us not trained to fly a 747.  For those who are trained as pilots, the set up is fairly easy to understand.  First, many of the controls are duplicated to accommodate the co-pilot.  Additionally, controls are grouped according to how they are are used.  Navigation and control are in one area, while communications is in another area.  The layout of the cockpit makes it easy for the pilot to understand the controls.

While it’s never been compared to a plane’s cockpit, the most frequent comment I get from staff at my library is that the home page is too complicated and too busy.  To me, however, the complexity and busy-ness should function like the airplane’s cockpit.  It should be laid out in an understandable fashion that gives the user not only access to the complexity buried beneath the home page, but an understanding that there is a complexity.  That there is a wide range of information available.  To find just what you are looking around, you may need to poke around a bit.  This is not to say that everyone should have to poke around, there needs to be express lanes for users doing the most common tasks on the web site, but hiding the complexity of library only makes it more difficult for the user.

Current approaches to representing that complexity are far from perfect.  And new solutions that provide a single index that searches many of the library’s resources can help filter some of the complexity.  However, even these solutions mask the complexity more than they cure it.  To deal with the complexity of its results page Google gives you search options that narrow the scope of your search (Scholar, Blogs, Images, etc.).  However, those categories still fail to represent the complexity or give us enough options.  So we are left to hope what we are looking for is in the first few pages of results – to Google’s credit, it often is.

Library web pages lack one of the strengths of cockpit design – standardization.  Pilots can move from plane to plane and expect similar layouts.  In libraries, for now, our users have to learn a new interface every time they come to a new library home page.  And the incentive in libraries is to come up with something new rather than make user experience uniform from interface to interface.  It creates a challenge for our users and it creates a challenges for the librarians who maintain web pages.

Dr. Norman also pointed out that there are many things we don’t expect people to be able to do intuitively, such as driving.  We expect people to spend some time learning to drive, perhaps even taking classes.  Although it may be unfashionable to say in some circles, doing research is a skill that has to be developed over time.  Obviously, the stakes (usually) aren’t as high as they are when you are driving a car.  But being able to identify and meet your information needs is valuable skill that we spend all of our lives practicing.  Unfortunately, research doesn’t have the clearly defined rules and standard tools that driving does.  This makes it even more complex.

I want to be clear, I’m not coming down on the side of librarians who I have known who thought our users just need to learn how to use our stuff.  Users do need to learn how to do research, but librarians need to work towards making our tools more consistent and easier to use.  To go back to automobile example, manufacturers keep making better cars; standardizing the order of the gear shifter, anti-lock brakes and now cars that park themselves.  Just because the user has a responsibility to learn, does not relieve librarians of the responsibility to make better tools for research – or to teach users how to do research.  Ideally, librarians would develop a tool that would teach its user how to do research while .  I’m sure someone somewhere is working on these issues, but I don’t see it mentioned frequently in the Library literature.

The library domain is a complex one.  The array of resources that libraries make available to their users is staggering.  We need to do a better job of getting our resources in the hands of our users.  To do that we have to work WITH our users to help them understand what information they need and how they can use it.  We can’t abandon them to the single search box of Google or Bing!

Let Drupal Drive Your Web Site

Below are my notes for the Poster that I will be presenting at the ALA Annual Conference in Chicago on July 13, 2009.

The data and charts that accompany these comments are available for download.  As is the raw survey data.

Who was surveyed and when?

  • Libraries that are using Drupal for their web site or planning to use Drupal for their website were surveyed from May 1, 2009 to June 15, 2009.
  • 71 libraries completed the survey.

Type of Library

  • Academic libraries represented the largest group with 37 University, College, or Community College libraries responding.
  • 26 Public Libraries responded.
  • The remainder of the libraries were made up of a vendor, a consortium, Government, Special, School, and Medical Libraries.

Before Drupal

  • More than half of libraries were using Static HTML with some dynamic content for their web sites.
  • Libraries using only Static HTML or Static HTML with dynamic content make up more than two-thirds of the libraries that responded to the survey.
  • Libraries using a homegrown CMS were the next largest group.

When was Drupal implemented?

  • Of the libraries that responded, the bulk of the libraries were either still in development or had implemented Drupal more than a year ago.
  • Libraries who responded could be broken down into 3 nearly equal categories:
  • Libraries who had implemented more than a year ago.
  • Libraries who had implemented during the previous year, and
  • Libraries still in development.

Functionality

  • CCK and Views, Classification, Search, and Syndication/Aggregation are the most popular functions according to respondents to the survey.
  • Public and Academic libraries generally agreed about how popular each function was.
  • Notable exceptions
  • User-contributed content which 8 public libraries had implemented versus 3 Academic libraries
  • Multi-user and Personal blogs (14 Public v. 6 Academic)
  • Rich text editing (20 Academic v. 10 Public).

How hard were these tasks?

  • “Learning Drupal”, “Changing the site’s look and feel”, and “Module Selection” were reported as the most difficult tasks in Drupal.
  • “Creating and marking up content”, “Configuring access rights”, and “Installing Drupal” were reported as the least difficult tasks in Drupal.
  • “Getting help in the forums” and “Contributing to Drupal” were also rated as relatively easy.
  • “Finding skilled Drupal Developers”, “Finding skilled Drupal Designers”, and “Finding good documentation” were reported as challenges.
  • In general, Drupal was rated relatively easy
  • No task was reported as difficult by more than 50% of the libraries.
  • 3 tasks (“Creating and marking up content”, “Configuring access rights”, and “Installing Drupal”) were reported as easy by more than 50% of the library.
  • Close to 80% of the libraries reported “Creating and marking up content” as easy.

Who is responsible?

  • The Academic libraries who responded to the survey split almost evenly between having a single person responsible for the use/implementation of Drupal and having the Library’s IT department take responsibility.
  • Public libraries were more than twice as likely to have a single person assume responsibility for the use/implementation of Drupal over having a Library IT department take responsibility. This may reflect the availability of an IT department in the library more than any preference.
  • Similarly, assigning responsibility to a Committee or Task force was nearly twice as likely in an Academic library than in a Public library.
  • Assigning responsibility to a Committee/Task force was reported by one-third as many Public Libraries as reported that a single person had responsibility.
  • Academic libraries reported assigning a Committee/Task force responsibility half as often as an individual.

Challenges

  • The steep learning curve of Drupal was the by far the most frequently mentioned challenge in implementing and using Drupal.
  • Concerns about staff resistance, staff understanding of the new architecture, and lack of communication were also common themes in the responses.

Benefits

  • Decentralizing the content, ease of updating content and freeing up the time of the programmer/webmaster were frequently mentioned as benefits of using Drupal.
  • Many libraries also mentioned increased functionality for their web site as a benefit of using Drupal.