Interview - Newsfilm online, digitising ITN and Reuters archives

Jisc: Could you explain who you are what you do and explain a little bit about the project? 

Jeff Hulbert (JH): I am the project manager of Newsfilm Online. It’s a project which is intending to bring to the UK Higher and Further Education community access to 3,000 hours of news film taken from the ITN and Reuters archives. It will provide supporting metadata offering contextual information as to what the film is and what it describes and shows. We will also be bringing forward of the order of 450,000 pages of studio scripts and running orders. 

The studio scripts and bulletin running orders, covering the period 1956 – 1978, will enable the project to provide a complete rundown of every news story covered by ITN during that period. The period after 1978   will be geared very much to describing the film packages that will be served. In addition, we are bringing forward some ‘raw’ newsfilm material from Reuters material that was supplied by Reuters to its syndicated newsfilm customers in a pre-broadcast form. It is known as newsfeed or raw newsfeed and is usually edited into news ‘packages’ that are then broadcast within the customers’ news programme formats. Such material is not therefore compiled for broadcast in the first instance. This will provide an opportunity for UK Higher and Further education to look at film supplied to news companies and news providers by a news agency. It is pretty rare for academics to be able to gain access to such an asset. It is almost like getting access to the raw news feed that you get from news agency tapes or the press releases coming from government departments or third party organisations. So you can see what information goes into the editorial offices and then compare it with what comes out. 

We will be also digitising the entire surviving archives of a silent cinema newsreel, Gaumont Graphics (material survives from about 1920 until the newsreel stopped production at the end of 1932) and a sound cinema newsreel, Gaumont British News (1934-1959). Some material from other cinema newsreels, British Paramount News and Empire News Bulletin and Universal News will also be included. Reuters activites as a television news agency, date back to the second half of the 1950s and beginning with a newsfilm agency called BCINA [British and Commonwealth International Newsfilm Agency]. That developed in the early 1960s, in partnership with several major world broadcasters, into a news agency called Visnews.   Eventually in the early 1990s that became ReutersTelevision. Reuters Television camera teams are sent to cover news stories world-wide; for instance you may recall that one suffered a well-publicised fatality in Baghdad during the 2003 Gulf War.  

Jisc: It seems like a vast range of material for many subject areas. What criteria are there for choosing the material? 

JH:   There are criteria that we have actually tested during our scoping study. We have established broad criteria in the final report of the project scoping study which was delivered in August of 2003. We have now refined those further, having discussed them fully with the project’s steering group. That comprises around a dozen leading academics and is chaired by the former Chief Executive and Editor-in-Chief of ITN, Professor Stewart Purvis, of the City University. 

The broad criteria are that we will select material that comes from ‘major’ news bulletins such as News at Ten and Channel Four News. We are looking to go right back to the early days of ITN. It is easy to slice it up and say we will take four hundred hours from here, five hundred from there, but much more difficult the closer you get to the material, after all the archive has hundreds of thousands of hours of material to choose from. Of course we must also recognise various developments in ITN’s work, such as the introduction of News at Ten in July 1967, which was a new departure for television news in the UK and had an enormous influence. 1982 is another notable date because Channel Four News started in November of that year. That is a 50-minute news bulletin and it has developed into a programme that takes a different approach to news reporting and has much longer stories, such as Nicholas Glass’ arts reports, which can last up to ten minutes. 

We have also discussed with the steering group the finer grain selection and agreed that one of the criteria to be applied in selecting material will be driven by whether ITN or Reuters own the material’s copyright. It may not be appreciated by everyone, but much of the footage broadcast in television news programmes is sourced from news agencies and other broadcasters; and they retain the copyright ownership of the pictures. The licence signed by HEFCE, ITN and Reuters covers only the newsfilm material in which ITN and Reuters hold the copyright. So, where there are copyright restrictions on some material that will tend to exclude those stories from selection. However, for stories that are regarded as ‘iconic’: those that we could not possibly omit, but in which there is some   third party material, we have got two choices. The first is to come to an arrangement with the rights owner to be able to bring that material forward. The other option is to exploit a feature of our encoding system that gives us the opportunity to ‘fuzz out’ material that isn’t owned by ITN. So if there is a two minute news story that includes, say, ten seconds third-party copyright footage that we cannot economically achieve rights clearance for, then we can blank that out and substitute a caption, whilst maintaining ITN’s commentary on the soundtrack. 

In the finer grain we are also going to be taking vertical slices from ITN’s coverage so that while in the generality the project may concentrate on News at Ten, ‘slicing’ will enable material to be drawn from a whole week’s output. The sampling frequency covers the first full week of each quarter for a whole year; and every fifth year is selected, starting with 1956. Using the first full week of every quarter will potentially avoid some calendar fixtures. We will sample ITN material covering all the bulletins in the particular week. 

The rationale for doing this is that if we are concentrating on one bulletin (‘horizontally’) we would otherwise miss out on the different treatment that news stories were accorded by different bulletins. For instance, Lunch Time News has one type of audience, whilst the News at Five onwards has a different type of audience; and the evening news has yet another audience. By having a vertical slice for a week you can actually get not only an idea of the way the story has developed in a day, but also you can look at the different treatments. An interesting example on our trial website demonstrates this. Typically ITV news uses different reporters for ITV and Channel Four News programmes. However, Bill Neely filed reports about the 1993 Mississippi floods for Lunch Time News, News At Five, Channel 4 News and News At Ten. 

However, not only are we going to be providing the ‘clean feed’ packages – these are packages that were broadcast – but are archived without their on-screen captions. We will also, from 1987, up to the present be able to provide some examples of the programmes as broadcast which will show the studio sets, fashions, styles of presentation and how stills are use and so on. They will therefore provide illustrations of how ITN’s news broadcasting developed over the years. 

Jisc: To what extent will the slices reflect the way the material might be presented to the users in education? 

JH:   The whole point of presenting this material is that it can by packaged on the web dynamically and in an infinite number of ways. Newsfilm works on many different levels and over time it may be that material that might initially have seemed only to be of interest to one academic discipline may in reality appeal progressively to others. For instance, street interviews (‘vox pops’) might initially only seem to be of interest to those wishing to pick up ‘popular’ views about the subject of the interview. However, they act as records of dress (so potentially of interest to those looking for dress styles), hair styles, speech patterns (and slang) as well.  

Jisc: Did members of the steering committee or project team members ever consider a pay-per-view model? 

JH: I presume that you mean as a way of handling third party rights issues? That has not been considered, but it might be a way forward, but that is a question for Jisc to consider, since that falls within their purview. 

Jisc: In order to establish what education wants, are you conducting focus group and other activity? 

JH: We have an education officer who is engaged in education outreach and he is out on the road giving presentations about what Newsfilm Online is going to be providing, and collecting feedback for the project’s Steering Group to consider. The Steering Group comprises academics who are experts in their field. This enables us to investigate other ways in which we can present material in the first place, and the issues relating to that. 

Jisc: Referring to the digitising process itself, how are you going to ensure that it is technology proof for the future? 

JH: There are two issues. First, in taking material from the archive, a lot of it has had to be converted into a format that is compatible with our coding system and capable of being ingested. A vast amount of the material in the archive is in redundant media formats, 16 or 35 mm film as well as non-current videotape formats. ITN is also embarking on its own programme of migrating material from the older to modern current and digital formats. 

The second point is that we are producing two encoding outputs. The first of these is digital sub-master, which won’t be served on the web. It is an MPEG2 file. Conversion from this to other digital video file formats is a software based operation. For example you can convert it by a process called transcoding, to an MPEG4 file, or to QuickTime or Windows Media Player or other file formats. Such formats are dynamic products and I think it is inevitable that in five or ten years time these formats will be out of date and will need to be transcoded into whatever version is then current. New Transcodings, however, might not need to be constrained by the parameters within which our project is framed, since delivery technologies may have higher capacities and be faster that they are currently. Our project has to be mindful of what can be served across the web at its current state of technology. Broadly speaking it means that when we are encoding long film clips we have to split them up into bite sized chunks, of roughly three minutes duration. This is because video files – even the compressed video files that we are creating in the project – are large and can take a long while to download if delivered in one piece. The bite-sized chunks are easily deliverable across the web. The tree minute breaks are tempered by where natural breaks appear in the film, so you don’t snip into words or in the middle of story that may last another thirty seconds. You can download 5 or 10 megabytes of material relatively quickly, but you can’t do this with files if they were of a hundred or more megabytes. 

As Windows Media Player and QuickTime are successively replaced by newer versions, you may find that either the available delivery bandwidth increases or that the programs benefit from new compression technologies that could result in smaller deliverable files. In both cases it is possible that the segmentation decisions that we have taken might need to be reviewed as well. For instance, ITN’s Roving Report, running at around 26 minutes, will be delivered in around seven or so 3-4 minute segments. In five or ten year’s time, it might be entirely normal to be able to deliver 10 minutes or so of video. In that case you will need some human intervention to re-segment those films. 

Jisc: Is someone actually watching these clips as they are encoded. How is the metadata applied to ensure that searches return the right result in the end product? 

JH: The encoding is being carried out by a digital asset management system, so metadata and clips are associated within that. When the material is exported from that to the Jisc data centre, there will be indexed links between clips and metadata, so it will be clear what metadata relates to which film. Those links will be robust. As regards sampling of the video encodings, those that have to be segmented will be viewed in their entirety, since the natural breaks can only be identified by human intervention. In the case of newsfilm packages that were prepared for broadcast in news bulletins, a lot of those clips are under three minutes duration, so our policy there will be to sample a representative number; and that could be anywhere between ten and twenty percent of those clips. There is simply not the time to sample the whole lot, as we are talking about 3,000 hours running time being created over the remaining 18 months of the project. 

Jisc: Who makes the decision about what particular subject area a clip might relate to? 

JH: Cataloguers create records by looking at a combination of programme diaries and ITN’s existing databases to identify the stories. There is summary information already in existence for a large number of the stories. If there is already metadata it will give a flavour for what the story is about that will be for selection. However, the project’s Senior Researcher and I will provide day to day guidance; and that will be referenced to the criteria endorsed by the Steering Group. Broadly speaking – and irrespective whether the story is about art, science or politics – one of major determinants will be whether the copyright of the footage is held by ITN or Reuters. 

Jisc: What stage have you reached within the project now? 

JH:   Our encoding system is being set up. We will be road testing it for a month – acceptance trials, if you like. This will show whether there need to be tweaks to the database, to the codecs or whatever. It will also show us how many hours of encodings per day we will be able to achieve. Some of the newsfilm will be ingested from compilation tapes that we have created: where material has been transferred directly from film onto video tape. These typically will comprise an hour’s running time. Other material, however, is already held on video tape in the archive and will be ingested directly from those copies. So to produce an hour’s running time encodings from those sources will take longer as several tapes will need to be inserted and ejected from the tape decks. Feeding the tape decks is a hand-operated process.   

Jisc: how long do you expect it will take before material is delivered? 

JH:   A small amount of material is already being delivered via the demonstrator website. There is another thirty hours of scoping study material available for delivery. That can provide potential users with a greater experience with what is there. 

Jisc: What do you think is the benefit to the partners involved with the project? 

JH: Quite a lot. This is a partnership that really does demonstrate a win-win. In my view partnerships, if run well, don’t appear to be partnerships, but friendships or collaborations. 

The duration of the licence signed by HEFCE, ITN and Reuters is in perpetuity. From the ITN and Reuters perspective, it is putting the material into the academic domain. Academia is not some separate community; it is part of the greater UK population, a consumer of television programming, although it is a specialist audience. I cannot presume to speak directly for either ITN or Reuters here, but it might mean that there is potential for sales of film material. In addition, the project is discovering new material – or in some instances just looking at known material with fresh eyes – selecting it and having the material transferred from older media on to current media formats. That is then available for sale by ITN or Reuters. 

The companies are also receiving some income for the rights in the material selected for delivery via Newsfilm Online under the terms of the agreement with HEFCE. Also, as a project, we are discovering material that the archive has not had the additional staff resources to discover for itself. Commercial archives operate under considerable time and financial constraints. We are completely free to choose within the limits of our own selection criteria where in the archive to go and what to look at. We are able to bring material forward such as the broadcast Roving Report programmes, that have laid in film cans in the archive for over forty years, but have not been fully investigated or exploited before now. 

Jisc: Those areas, like the iconic areas that you mentioned, they will probably be looking at how you overcome those issues. 

JH: One-to-one negotiations with third party copyright holders will be helpful, although tempered by the fact that Newsfilm Online is not intended to be a comprehensive encyclopaedia of television news. For instance, there is quite a lot of Vietnam footage in the archive that that has come from US sources. It might be helpful if we could achieve an agreement with the company to enable some of the material to come forward. So there might need to be some bilateral negotiation with those companies, conducted by the BUFVC’s Director, Murray Weston perhaps in partnership with some of our ITN colleagues, or maybe in collaboration with other UK education bodies. It is possible that negotiations could cover either specific pieces of film or material from particular sources in general. I feel it is worth pursuing both paths. Inevitably there will be some third party material within some of the stories that we select, but I would expect it to be a modest amount, because otherwise the material won’t be particularly reflective of ITN or Reuters. In such cases, if we have not secured third-party rights clearance we use the video captioning facility that I mentioned earlier. 

Jisc: Are you going to set up exemplars for people in education which illustrate how people will be using this material? 

JH: Yes – that is absolutely essential. There will be researchers, teachers and learners who will want to go and find clips for themselves. However, there will be lots of others who will not have the skills or time to spend in searching out material. The objective has got to be that it will work for both those types of users so that each will be able to get a meaningful experience from Newsfilm Online. The concept of packaging has been with us since the scoping study began. So packages initially created might include ten film packages from the outbreak of war in 1939, street fashion, natural disasters – such as floods and volcanic eruptions – scientific discoveries or scenes from everyday life. And the list can grow over time. Each would have an introductory guide to the material that provided some contextual information. This is an approach adopted by the National Archive and a number of museum websites.

Jisc: Do you think you use your contacts in ITN and Reuters to promote this?

JH: We have already been speaking to some ITN journalists, who are interested in the project; and then, of course, we are very fortunate in having the former chairman of ITN as the chair of the Steering Group. So, we have the potential to reach parts that others could not reach. 

Jisc: We look forward to hearing more soon.