Intranet Metrics – Discovery, Satisfaction and Impact

When I begin a new intranet project I am of course immediately interested in any metrics that have been recorded and reported on the performance of the intranet. In the majority of engagements I find that the lack of resources in the intranet team has meant that any metrics assessment is very limited indeed. This makes it difficult to know where to start in giving advice and the result is that a programme of user research has to be undertaken to give a credible baseline against which recommendations can be made. This takes time and increases the cost of the engagement.

My new Research Note on intranet metrics provides an overview of the ways in which the performance of an intranet can be assessed and can be downloaded from the Research Notes section of this web site.  It is based on Chapter 15 of my Intranet Management Handbook, published in 2011. The 26 page report covers technical performance, discovery performance, user satisfaction and business impact, and describes both qualitative and quantitative methodologies. It also highlights the difference between summative and formative assessments.  In addition to techniques that can be used directly by an intranet manager I have also summarised the methodologies used by five external intranet benchmarking services.

Quite a substantial amount of work has been undertaken over the last few years on assessing the performance of enterprise applications, taking into account the provisions of ISO 9241. The book by James Lewis and Jeff Sauro entitled Quantifying the User Experience: Practical Statistics for User Research, published in 2012, should be essential reading for intranet managers. Both the Nielsen Norman Group and Rosenfeld Media publish a range of reports and books on user experience assessment.

Inevitably there is a fine line between the user research required in the process of developing a new intranet (or a substantial re-design of an existing intranet) and the research required to optimise the operational performance of an intranet. Many of the techniques described in this report have a value in both situations. Feedback on any of the topics in the Research Note would be appreciated.


Martin White



Intranet Now Diamond Award 2015 – nominations now open

No one was more surprised that me to be presented with the inaugural Intranet Now Diamond Award at the 2014 conference. I have to admit I was not really concentrating on what Wedge and Brian were saying and suddenly realised that they were inviting me up onto the stage. I was incredibly honoured and for probably the first time in my career I was somewhat at loss for words. I’m now looking forward to the 2015 Intranet Now conference and having the pleasure of giving the Award to another of the many people who we all rely on for their wisdom and their networks.

The Intranet Now Award is unique in that it is awarded to an individual for their remarkable contribution to the community at large. Wedge and Brian are now seeking recommendations for the 2015 Award. There are of course many intranet managers who have made a significant impact on their organisation, often as a team of one.  However they are looking for someone who is committed to raising the awareness of good intranet practice amongst the wider intranet community in the UK. It’s not as if Wedge and Brian do not know potential candidates but they are firm believers in the wisdom of crowds so would like to know who you respect as an intranet guru.

So could you look through the list of the people you follow on Twitter and the blogs you monitor.  But please bear in mind they are looking for someone who participates in our community and not just observes it. To me there is one obvious candidate…..I wonder if you agree?  Information on how to nominate someone is on the Intranet Now site

Martin White

45 years of managing information

Forty-five years ago this week I started out on my career in information science. My first post was as an information officer at the British Non-Ferrous Metals Research Association near Euston Station in London. I could not have started in a better organisation. The BNF was a pioneer in information management in the period up to WW2 under a former Director, Professor R.S.Hutton and continued to invest in a large team of librarians and information scientists. My two colleagues, Clive Mitchell and Brian Perry, were alumni of the Centre for Information Science at what is now City University. Over the next three years I learned both the theory and practice of information science, including the use of 10,000 hole feature cards for information retrieval from a large collection of research and production literature. I wrote around 10,000 abstracts at the BNF and the emphasis was always on the quality of the metadata I applied rather than the quality of the abstract. Along the way I learned to speed read technical articles in English, French and German, a skill that has been of considerable use over the years!

When the BNF moved out to Wantage I stayed in metallurgy and engineering at the Zinc/Lead Development Association, running a virtual team of staff in seven offices around the world without the benefits of email.  My career then moved on into patents and trademarks, technology forecasting, electronic publishing, market research in the IT and telecommunications industries, knowledge management consultancy and finally intranet and information management consulting in 1999. Computer-based search services arrived in 1975 but  was not until I was  working at Reed Publishing in 1982 that email, CD-ROM and mobile phones all started to be a part of the technology mix, along with IBM pcs. I’ve lost track of how many companies have been clients since first venturing into consulting work in 1979, the same year I started up a newsletter on Information Management with Helen Henderson. It must be of the order of 300 or more, giving me business experience of 39 countries and the confidence to write seven books. Without doubt the most memorable project was working on an intranet strategy for the IMF in Washington; we started two days before 9/11 and finished on schedule. With the IMF as a reference client selling Intranet Focus Ltd services suddenly became much easier and led to projects for the United Nations and the World Bank.

I am very fortunate that not for a single day have I regretted my choice of career. I have been able to use my information science skills (and sometimes my undergraduate chemistry!) in every organisation I have worked for, though often in quite different ways. That has been immensely satisfying, especially since the fundamentals of effective information management are the same now as they were in 1970. Focus on understanding user requirements before providing information, make sure that the information quality is as high as possible, and follow up to learn from the way in which the information was used. My only disappointment is that organisations still fail to appreciate the value of information and the impact that information management can have on business success. So there is still plenty of work to do!

Martin White


Digital Success or Digital Disaster? – Mark Morrell offers guidance

There are very few books on intranet management so this book from Mark Morrell, courtesy of the Intranatverk organisation, is very welcome. However the scope is broader than just intranets. The title indicates that it will also be useful for web sites and digital workplaces. Mark has a great deal of experience as an intranet manager, both at British Telecom and then as a consultant, and his practical experience at ensuring intranets have high quality content and a focus on business requirements comes across strongly in this e-book. In many instances he quotes (anonymously!) from some of the projects he has worked on. The six chapters cover strategy development, governance principles, governance frameworks, governance hierarchy, publishing standards and finally the benefits of adopting a strong intranet governance approach. The book runs to almost 200 pages and so is a substantial contribution to the intranet literature.

The book is very much a personal statement by Mark on the importance of governance frameworks and content quality standards in intranet management. I would have appreciated references to other books or web resources on the subject, especially with regard to SharePoint intranet implementations where attention to governance detail is absolutely critical.  I would also have liked to have seen more checklists as a way of making sure I had fully understood the guidance. They would also provide a framework for prioritization and action with the intranet team.

There is much of value in this book but you have to work a little too hard to find it. This is because although each chapter runs to between 20 and 30 pages but there is no detailed list of chapter contents and no index. As a result you have to scan through each chapter, a process not helped by sub-heading typography that does not give a strong sense of structure. The page layout is fine for a printed version but in pdf format the change in margin width from even to odd page numbers is also distracting, especially when scanning through at speed.

Don’t let this discourage you from buying a copy at just £10.99. There is no other book (even mine!) covering governance issues in such detail and even experienced intranet and web managers will benefit from working through the book screen by screen.

Martin White

Enterprise Search Europe 2015 – the first delegates have registered

As a conference developer there is nothing more welcome than registrations as soon as the programme has been launched, and that was the case last week with Enterprise Search Europe 2015. The conference takes place in the Olympia Conference Centre on 20-21 October and the programme has more case studies than all the other papers put together. This year the programme committee spent some time getting a balance between case studies, papers on strategy and planning, and roundtable sessions where delegates can exchange views and business cards. We also made sure that we have a balance between open source and commercial search case studies. There is a lot of interest in open source search and that is why the keynote this year is being given by Charlie Hull, the founder of Flax. However in terms of installed base in enterprise applications the balance is still towards commercial applications, in particular SharePoint 2013.

One of the innovations this year has been scheduling round table sessions within the conference programme. The sessions cover open source search, SharePoint 2013 search implementation and the use of search logs to prioritise search development work. The timing is such that delegates with a specific interest in these topics are unlikely to be as interested in the parallel conference papers.

We are especially pleased to have Miles Kehoe (Avalon) on the programme this year. Miles is probably the most experienced search implementation consultant around, having started in the days before the launch of the Verity enterprise search application. More recently Miles was a senior member of the Lucid Words team so brings practical knowledge of both commercial and open source solutions to the programme. As well as a presentation on critical success factors for search success Miles will also be closing the conference with his view of how enterprise search will develop in 2016.

The timing of the conference has been shifted this year from April to October specifically to provide delegates with a perspective on options, opportunities and challenges in 2016 as input into business planning for year ahead. However in addition to the conference papers we will be making every effort to bring people together with common interests, not only in the workshops but throughout the conference and the social evening on 20 October.

Martin White

Cognitive barriers to information seeking

From time to time I turn into a Visiting Professor and work my way through recent papers in academic journals. One of the core titles is the Journal of Information Science and in the Online First section I have just come across what is probably a seminal paper on cognitive barriers to information seeking. The author is Professor Reijo Savolainen from the School of Information Sciences at the University of Tampere. The School has world renown for the quality of its teaching and research.

The author addresses two challenging issues; the conceptualization the features of cognitive barriers to information seeking and the characterization of the impact of cognitive barriers on information seeking. This was undertaken by a very thorough review of the literature, and the bibliography extends to over 50 citations.

From the analysis six barriers are described.

  • Unwillingness to see needs as information needs
  • Inability to articulate information needs
  • Unawareness of relevant information sources
  • Low self-efficacy, where the user feels that it will be difficult to obtain the documents
  • Poor search skills
  • Inability to deal with information overload

From my own experience working as an enterprise search consultant none of these barriers would be recognised by what is often a technology-led enterprise search team. The objective of the team is to develop “intuitive search”,  seeking to emulate Google web search in the enterprise without understanding the futility of doing so.The issue is not one of information retrieval but of information management, and the need to train employees in how to manage their personal and professional information life-cycles. Picking up on the low self-efficacy issue, there was a paper given at the CKIM conference last year which raised the issue that relevance assessments do not take into account a user’s perception that a document is going to be difficult to obtain (the JIS paper above is behind a pay-wall) that it is ignored as a ‘relevant’ result.

The author comments in the conclusion to the paper that a limitation of the study is that it approaches the cognitive barriers from the viewpoint of an individual actor seeking access to information and there is a need for expanding the research perspective by making use of ideas framed in practice theories, for example, because they would take into account how cognitive barriers are constructed socially within work teams and communities. It is easy to focus on the obvious issues around poor search performance (primarily content/metadata quality) that we do not take into account more complex issues around cognition which require search managers to be mind-readers, or at the least have a background in cognitive psychology.

Martin White

Management information and ‘conduct risk’

Deloitte has recently published a report entitled Management Information for Conduct Risk – Underpinning Better Decision Making. According to Deloitte ‘conduct risk management information’ covers market propositions, conduct, behaviour, culture, breaches of policies or regulations and the effectiveness of conduct risk mitigants and controls. Now the primary focus of conduct risk is the effective management of financial services market but the principles are, in my view, applicable across all organisations.

The report sets out ten elements of a conduct risk MI and these are

  1. Linked to strategy, culture and risk management framework
  2. Outcomes focused
  3. Holistic and used to support analysis of trends
  4. Forward-looking
  5. Efficient and proportionate
  6. Accurate and timely
  7. Measured and reported on at an appropriate frequency
  8. Comprehensive and traceable
  9. Supports open communication and challenge
  10. Acted upon and recorded

That is a very good set of elements for any information management strategy. All too often ‘management information’ is seen as the output from a business intelligence application. This report emphasises the need to integrate data and information, and also the need to have staff with appropriate skill sets. When I speak to senior executives I always ask them how much confidence they have in the reports that they are presented with on organisational performance. The answer is always that they have total confidence but when I probe they often have no knowledge of where the data and information has been sourced from. Many years ago I was working on the intranet of the International Monetary Fund and had an opportunity to look at the documents that were going to the Executive Board. Almost every item of information was tagged with the name of the expert taking responsibility for it. Good intranets take the same approach but many other enterprise reports have no personal accountability stamped on them simply because there is no appropriate field in the database.

New regulations from the Bank of England will require a much higher level of responsibility by managers for the decisions they take and the basis on which those decisions are made. Of course this is at present a specific requirement of the financial services sector but it should also be good practice in all organisations where shareholders increasingly want to know why a particular decision was made. Even if you are not in the financial services sector this 20 page report is well worth reading as a way of starting discussions in your organisation on personal responsibility for information quality. I’m always looking for ways in which I can sell information management to organisations and this is an approach that I can see having some merit.

Martin White

The Organisation in the Digital Age

Jane McConnell (NetJMC) has just released The Organisation in the Digital Age, the 2015 edition of her global survey of the way in which digital workplaces are being implemented. A good place to start this review is with a paragraph from the preface to the report. “Every organization has a digital workplace today. Every organisation has digital platforms, tools and services, that are more or less well organized and coordinated. Every organization has work and management practices, different cultures and leadership styles. In all organizations people work together – sharing information, taking part in projects and collaborating. All these elements combine to form the digital workplace: the intersection of people, organisation and technology”.  Absolutely!

It is virtually impossible to review this report in a conventional sense. The analysis is based on replies from over 280 organisations around the world, which as far as I know remains the largest such survey of digital workplace trends. Jane is also in the position of having a wealth of information from surveys carried out over the last decade, and many organisations have participated on a regular basis. The survey outcomes are presented in the 108 figures and tables. This year Jane has commented herself on some of the trends. This is a valuable innovation as in the past the report was faithful to the information from respondents but now there is the added benefit of Jane’s insights based not only on the survey but on her many consulting engagements.

Another innovation this year is the use of a portrait format for the pages, which for me makes it easier to read on a screen, and the graphics quality is even better than in previous years. The opening section looks at the progress achieved over the last year. This is followed by a section that assesses developments through four business scenarios.

  • The learning scenario – how easy is it for people to learn and develop their skills
  • The customer scenario – what is the level of support provided to customer-facing staff
  • The agility scenario – the extent to which the organization can react to major events
  • The knowledge scenario – how can the knowledge of employees be retained when they leave

There are then three chapters looking at some specific aspects of digital workplaces, including the importance of trust, the collaborative and social dimension, and the extent to which organizations need to embrace an open and informed culture. Then comes a chapter on leadership and transformation, which for me is probably the most important single element of the report, as the maturity of a digital workplace is directly related to the extent to which senior managers support the initiative. Finally comes an analysis of the drivers for the adoption of digital workplace principles and practices.

This 130 page report is priced at £360 plus VAT (€490 or $550) and this is a site license so that you can make multiple copies for your colleagues. This is important because the value of this report is significantly enhanced by being shared and discussed, perhaps even page by page and figure by figure. Even if you are currently sceptical about digital workplaces this report will help you decide whether now is the time to think about changing your mind. The report offers no easy solutions, or a list of “10Things To Do To Be Digital”. The advice is embedded throughout the report and put into perspective through Jane’s own comments on many of the issues. My digital library has almost 100 reports and papers on digital workplace development but none come close to emulating the information and wisdom of this report.

Martin White


The pleasures of being an intranet consultant

The last two months have been the busiest since I set up Intranet Focus Ltd in 1999. I have been working on two global intranet projects at the same time, with some similarities (e.g. a potential migration to SP2013 and almost total user dissatisfaction with search) but in other ways quite different in their current situation and in business drivers.  I have been working on one project for some time and then Sam Marshall (Clearbox Consulting) asked me to join him in bidding for a project where our combination of skills would be a winning proposition. The client certainly agreed and even better it’s been a win-win for us as we have shared out the tasks and introduced each other to some of the ways we each approach intranet management. Of course Sam and I have known each other for around a decade but this has been the first time we have worked together, and I hope it won’t be the last.

The first pleasure of being an intranet consultant is that there are always new things to learn. When I started up Intranet Focus in 1999 the chairman of the business I was working for warned me that all the issues about intranets had been identified and solved, as by that time there were probably 6 or more good books available on intranet management. Luckily he was wrong. Good practice (there is no such thing as ‘best practice’!) is constantly changing as novel business requirements emerge and new technical solutions become available.

The second pleasure is being able to talk to people across the organisation. For the project with Sam that meant interviewing over twenty stakeholders across the organisation. I always start off a stakeholder interview project reading through Steve Portigals invaluable book on interviewing.  This gets me back into the groove as I write myself a list of do’s and dont’s, both of which are easy to overlook in the pleasure of listening to people talking with passion about how important information and knowledge are to achieving business and personal objectives.

The third pleasure is being able to provide solutions to current challenges which can be implemented at little cost and effort. In our joint project Sam and I made some recommendations in an interim report which will probably have been implemented by the time the final report is signed off. There is also a related pleasure in helping an intranet team realise that what seems to be reasonably easy (migration to SP2013) may in fact be more challenging than IT are anticipating as it would be an invaluable opportunity to improve content quality and in particular metadata consistency.

The fourth pleasure is in being reminded that great organisations have great employees who have an understanding of the business and a commitment to success that we have to match as consultants. What ever the technical solutions for an intranet there are always two groups of users – content publishers and content users. It is so easy to overlook the demands on content publishers. “We are a part of a global organisation, so people often have to work late to talk to colleagues in the USA and Canada. As they leave at 8.30pm there is no way I can ask them if they have been able to update our department profile.”

So this is my first blog post since the second project started, but I have no intention of reading through several thousand Tweets that have passed by unnoticed, un-forwarded and un-commented. Normal service will now be resumed… least for the next few weeks.

Martin White

Enterprise Search Europe 2015 in London on 20-21 October

A couple of recent search conferences have caused me to think carefully about what makes a successful event for search managers. At both the Findwise Findability Day in Copenhagen and at the Smarta Sök event in Stockholm it was very clear that the delegates were especially interested in gaining experience from case studies. Although there are many technical innovations in search, for most organisations the objective is to find out how to provide employees with a better search experience with the minimum about of near term investment. When I sat down with the Information Today conference team a few weeks ago to talk about the 2015 Enterprise Search Europe conference we decided to fill it to the brim with case studies. The outline programme gives us space for at least 15 case studies over two days, together with three short round-table sessions that will run in parallel with the presentations. The theme is ‘Delivering Search Excellence’ both from the content of the papers and the conference event overall.

This year the conference has been moved from the Spring to the Autumn and is co-located with Internet Librarian International at the conference centre in Olympia in West London so there will be quite a buzz around the venue. There will be a social event on the evening of Tuesday 20 October.

The call for papers is now open, and I’d encourage anyone with a case study to consider presenting it as a paper. It doesn’t matter at all if the case study is work in progress. Search implementation and enhancement should never be a project but a continuous process, and there will be lessons to be learned even from the initial stages of an implementation. Remember that you will be speaking to an audience of very understanding delegates who will be able to benefit from whatever experience you have to offer. We would like the case studies to be about real organisations, rather than ‘a global manufacturing company’. The company context is very important in understanding the value of search and also the implementation and management challenges.

The papers will be short – just 25 minutes with a 5 minute Q&A and we will be arranging the venue so that you will be able to continue to talk to the speakers at the end of each session. I should also add that the term ‘enterprise search’ also applies to corporate web site search, and there are often lessons to be gained from how search has been managed in a public-facing site. We would also be interest in papers from organisations that are using the search features of intranet software packages.

You may recall the words of David Bowman in the book of the film 2001 – a Space Odyssey to the effect that the monolith was full of stars. We want delegates to ESE2015 to remark that it is full of case studies!

Martin White

Chair, ESE 2015