Net4MRLW

Introduction

Net 4 MRLW is a bundle of scripts written in Python. These are not of supremious intelligence nor unavoidable difficulties but make the everyday work of social scientists easier.

These scripts are developed as a module of the platform of sociological investigation Marlowe (which codename is MRLW). Mrlw is part of the Prospero Suite. The French acronym Prospero stands for Software for the Pragmatic, Experimental and Reflexive Sociology Program and takes advantage of the fact that the French word for Software and Scientific Program is the same (Programme).

Despite Net4MRLW is developed as a module of the MRLW investigator tool, no Prospero Software is concerned by this Sourceforge webpage. Nor Caliban, neither Marlowe are indeed free Software, they are not intended to be and won't be commented on this webpages. This Sourceforge space is dedicated to Net4MRLW, and more particularly on issue concerning

• retrieving information from the Internet through search engines and freely available database

Net4MRLW is developed as a cross platform script. It is known to work under Linux and Windows.

Installation

Just download the Net4MRLW archive from the SourceForge releases page. Uncompress it in the folder of your choice. That's it!

How uncompress archive files

• On Linux, you can untar a file with the following command line

  tar zxf /path/to/the/file.tar.gz

  
  Note that recent Linux distributions provides GUI for this operation (such as FileRoller, for instance).
• On Windows, you are advised to use 7-zip. Just right-clic the archive file and choose "uncompress".

How to choose the location

• For most people, just any folder location is good.
• Those who aimed to use this bundle of scripts with the Marlowe Software should uncompress the archive under their mrlw directory (usually, under Windows' Program Files\DOXA\prospero\mrlw Folder).

Net4MRLW is a Python script

For most of the users, a working Python will be needed to interpreted the scripts. Anyway, for some privileged users, installing Python won't be necessary.

Command line uses

For a command line use, you absolutely need python installed on your computer.

• Under Linux Debian, simply ask root to

  apt-get install python

  (this is not necessary on the big majority of Linux boxes, given the fact that Python is usually already installed).
• Under Windows, ActivePython should help you.

Marlowe-companion uses

Those who aim to use this bundle of scripts with the Marlowe Software needn't the Python Language installed on their computer. Instead, they need (of course) a properly installed instance of Marlowe. Thank to its genius programmer, MRLW will interpret the python scripts by itself (!).

Access a search engine - PyGoogle

Just download the PyGoogle package from the Sourceforge server and uncompress it in the folder where you have extracted the Net4MRLW scripts.

To use this (core) feature, you'll also need a identification number to the Google API. You ll be provided with such a number on the Google API Webpage.

When you have this number, just write it down in a text file (named googlekey.txt) in the same location as your Net4MRLW scripts.

PDF Conversion - xpdf

For PDF access and conversion, Net4MRLW use the pdftotext utility included in the xpdf package. Make sure that pdf2text is in your Path.

• Under Linux Debian, xpdf is usually part of the base installation. Installing xpdf utilities is as easy as typing (as root)

  apt-get install xpdf-utils

  . Again, under Debian, no further operation is needed. Under a different distribution, you will perhaps have to include the location of pdftotext in your path.
• Under Windows, you'll have to install xpdf in order for the pdf conversion to work nicely (no pdf-to-text converter other than xpdf is supported at this stage). Download and extract the xpdf archive in in the folder of your choice. Make sure that pdftotext (the folder you choose) is in your Path.
  • On a old dos-based system (including Windows 95/98/ME), you can modify your path directly in the autoexec.bat file by inserting the line

    set PATH=%PATH%;c:\your\xpdf\folder

    in your autoexec.bat file. Don't forget to replace "your", "xpdf" and "folder" by the exact folder of your system where you have chosen to extract the xpdf utilities. having modified and saved autoexec.bat, you will need to reboot your system for the change to be effective.
  • On a Windows-based system (NT/2000/XP/Vista), your path can be modified in the menu Control Panel\System\Advanced\Environment Variables. Choose the Path variables in the second window (dedicated to system variables). A button allow you to edit it. Add a semicolon ( ; ) and the folder address (for instance c:\your\xpdf\folder ) at the end of the string. No reboot should be necessary.
  If you are uncomfortable with this path procedure or simply if you are not allowed to modify such features (because the administrator of the machine prefers to give you the user only privileges), you could simply paste a copy of the pdftotext utility in the folder where your Net4MRLW scripts resides.

How to use it

1. Open a command line box
   • Under Linux, open a Terminal or switch to a console.
   • Under Windows, choose Start\Program\MSDos_Command of, if unavailable, choose Start\Run and enter "cmd". A black window should appear.
2. Launch Python from the command line

   python

3. The ">>>" Python prompt appears.
4. Load the script

   from net4mrlw import *

5. Choose one of the available commands to launch a query on the Internet. For instance, "cherche" will invoke PyGoogle.
6. The result of Net4MRLW are saved in a txt-corpus on the hard drive.
7. As any Python command line session, you can quit the shell with ctrl+d.

To-Dos

1. Query through search engines ('cherche' command)
   • Diacritics management to be improved
2. Investigation on public persons
3. Investigation on co-occurrence
4. Definition of terms
5. Comparison of definitions
6. Syntactical/Pragmatical Pattern Matching
7. Encyclopedic information

Screenshot

Here is an example of use of Net4MRLW as a command line:

Author

The Net4MRLW project is headed by Christophe Lejeune. Feel free to contact him for comments, questions or suggestions. Any contribution welcome...!