BibEdit Admin Guide¶

1. Overview¶

BibEdit enables you to directly manipulate bibliographic data, edit a single record, do global replacements, and other cataloguing tasks.

2. Edit records via Web interface¶

To edit records via the web interface, please go to BibEdit Editor. This interface will let you add, change or delete fields in a record.

If you want to change several records at once, you can use the batch command-line techniques described below or the Multi-Record Editor.

Please note that 8564 tags pointing to fulltext files managed by Invenio can’t be manipulated via BibEdit. In order to modify them please use the FFT tags through BibUpload or the bibdocfile command line tool.

3. Edit multiple records via web interface¶

The purpose of the Multi-Record Editor Web interface is to allow cataloguers to easily edit more than one record in one go.

The Multi-Record Editor allows cataloguers to easily look up various records in the system in order to find record sets upon which to operate, and then to allow some easy replacement procedures on these records in one go, e.g. a substring substitution of some field value in some field tags.

Search criteria:    [         ]
Filter collection:  [         ]
Output tags:        [         ]
[Search]

Field
[ tag ][ind1][ind2]     [Select action[V]]

700__   Update Field
        [u] [Replace full content]
            [Ellis J.]
        when other subfield [u] is equal to [Ellis A.]

4. Edit records via command line¶

The idea is to download record in XML MARC format, edit it by using any editor, and upload the changes back. Note that you can edit any number of records at the same time: for example, you can download all records written by Qllis, J, open the file in your favourite text editor, and change globally the author name to the proper form Ellis, J.

You therefore continue as follows:

1. Download the record in XML MARC. For example, download record ID 1234:

   $ wget -O z.xml 'http://your.site/record/1234?of=xm'
   

   or download latest 5,000 public documents written by Qllis, J:

   $ wget -O z.xml 'http://your.site/search?p=Qllis%2C+J&f=author&of=xm&rg=5000'
   

   Note also that you can access history of records as covered in a access record history section below.

2. Edit the metadata as necessary:

   $ emacs z.xml
   

3. Upload changes back:

   $ bibupload -r z.xml
   

4. See the progress of the treatment of the file via BibSched:

   $ bibsched
   

   If you do not want to wait for the next wake-up time of indexing and formatting daemons, launch them manually now:

   $ bibindex
   $ bibreformat
   $ webcoll
   

   and watch the progress via bibsched.

After which the record(s) should be fully modified and formatted and all indexes and collections updated, as necessary.

5. Delete records via command line¶

Once a record has been uploaded, we prefer not to *destroy* it fully anymore (i.e. to wipe it out and to reuse its record ID for another record) for a variety of reasons. For example, some users may have put this record already into their baskets in the meantime, or the record might have already been announced by alert emails to the external world, or the OAI harvestors might have harvested it already, etc. We usually prefer only to *mark* records as deleted, so that our record IDs are ensured to stay permanent.

Thus said, the canonical way to delete the record #1234 in Invenio v0.1.x development branch is to download its XML MARC:

$ wget -O z.xml 'http://your.site/record/1234?of=xm'

and to mark it as deleted by adding the indicator ``DELETED’’ into the MARC 980 $$c tag:

$ emacs z.xml
[...]
 <datafield tag="980" ind1=" " ind2=" ">
   <subfield code="a">PREPRINT</subfield>
   <subfield code="c">DELETED</subfield>
 </datafield>
[...]

and upload thusly modified record in the `replace’ mode:

$ bibupload -r z.xml

and watch the progress via bibsched, as mentioned in the section 3.

This procedure will remove the record from the collection cache so that the record won’t be findable anymore. In addition, if the users try to access this record via direct URL such as distributed by the alert engine (record/1234) or via their baskets, they will see a message ``This record has been deleted’‘. Please note though that the original MARCXML of the record stays kept in the database, for example you can access it by:

$ python -c "from zlib import decompress; \\
             from invenio.legacy.dbquery import run_sql; \\
             print decompress(run_sql('SELECT value FROM bibfmt \\
             WHERE id_bibrec=1234 AND format=\'xm\'')[0][0])"

In some cases you may want to hide the record from the searches, but to leave it accessible via direct URLs or via baskets. In this case the best it to alter its collection tag (980) to some non-existent collection, for example:

$ wget -O z.xml 'http:://localhost/record/1234?of=xm'
$ perl -pi -e 's,ARTICLE,HIDDENARTICLE,g' z.xml
$ bibupload -r z.xml

This will make the record non-existent as far as the search engine is concerned, because it won’t belong to any existing collection, but the record will exist ``on its own’’ and the users knowing its recID will be able to access it.

P.S. Note that the ``bibXXx’’ tables will keep having entries for the deleted records. These entries are to be cleaned from time to time by the BibEdit garbage collector. This GC isn’t part of Invenio yet; moreover in the future we plan to abolish all the bibXXx tables, so that this won’t be necessary anymore.

6. Delete all records¶

If you want to wipe out all the existing bibliographic content of your site, for example to start uploading the documents from scratch again, you can launch:

$ /opt/invenio/bin/dbexec < /opt/invenio/src/invenio-0.90/modules/miscutil/sql/tabbibclean.sql
$ rm -rf /opt/invenio/var/data/files/*
$ /opt/invenio/bin/webcoll
$ /opt/invenio/bin/bibindex --reindex

Note that you may also want to delete the fulltext files and the submission counters in /opt/invenio/var/data subdirectories, if you use WebSubmit.

7. Access record history¶

Every revision of the metadata of a record is stored in the “history” table containing all previous MARCXML master formats of the record. You can access them via the bibedit command line utility.

To list previous revisions of record ID 1:

$ /opt/invenio/bin/bibedit --list-revisions 1
1.20080319193118
1.20080318172536
1.20080311020315

To get MARCXML of the revision 1.20080318172536 (record ID 1, revision date 2008-03-18 17:25:36):

$/opt/invenio/bin/bibedit --get-revision 1.20080318172536 | head -5
<record>
  <controlfield tag="001">1</controlfield>
  <datafield tag="037" ind1=" " ind2=" ">
    <subfield code="a">CERN-EX-0106015</subfield>
  </datafield>
[...]

To compare the differences between the two last revisions:

$ /opt/invenio/bin/bibedit --diff-revisions 1.20080318172536 1.20080319193118
--- 1.20080318172536
+++ 1.20080319193118
@@ -4,7 +4,7 @@
     <subfield code="a">CERN-EX-0106015</subfield>
   </datafield>
   <datafield tag="100" ind1=" " ind2=" ">
-    <subfield code="a">Photolab</subfield>
+    <subfield code="a">Photolab SOME TEST EDIT HERE</subfield>
   </datafield>
   <datafield tag="245" ind1=" " ind2=" ">
     <subfield code="a">ALEPH experiment: Candidate of Higgs boson production</subfield>
@@ -26,7 +26,7 @@
   </datafield>
   <datafield tag="650" ind1="1" ind2="7">
     <subfield code="2">SzGeCERN</subfield>
-    <subfield code="a">Experiments and Tracks</subfield>
+    <subfield code="a">Experiments and Tracks SOME TEST EDIT THERE</subfield>
   </datafield>
   <datafield tag="653" ind1="1" ind2=" ">
     <subfield code="a">LEP</subfield>