first-workflow-Playarists

Starting from the ERIH-PLUS index of Social Science and Humanities approved journals dataset

ERIHPLUSapprovedJournals.csv (downloaded 23/03/2023) we want to retreive all the publications belonging to one of those journals, included in OpenCitation Meta database (https://opencitations.net/meta#:~:text=For%20each%20publication%2C%20the%20metadata,and%20PubMed%20Identifiers%20(PMIDs).)

We are currently evaluating what would be the most efficient methodology to do that, given the size of OpenCitation Meta data dump (56GB, 8GB zipped).

Current options are:

• Iterating over ERIH-PLUS issn list and request all publications in OpenCitation Meta by means of its SPARQL endpoint. This would include using SPARQLwrapper python library
• Downloading the data dump and performing chunk operations (either reading the csv with pandas setting a chunksize parameter, using os library to iterate over the folder's files, reading directly the zip file using gzip library etc.) Note that the OC data dump has a row for each entity that is either a publication or a venue. At this moment we don't need publication information, so we would need to cut down the dataset to only have venues information in it.

Adding Open Access information to the dataframe

Input: Dataset mapping OpenCitation venue data to ERIH-PLUS venue data, dataset containing country and discipline information for each journal, list of Open Access journals from DOAJ

Output: Updated dataframe with an additional column indicating whether the journal is Open Access or not

Fetch Open Access information from DOAJ

• Query the DOAJ API or download the dataset to obtain a list of Open Access journals and their ISSNs. Create a dictionary of Open Access ISSNs

• Using the fetched data from DOAJ, create a dictionary with ISSNs as keys and Open Access status (True/False) as values. Merge Open Access information with the main dataframe

• Using the dictionary created before, create a new column in the main dataframe indicating the Open Access status for each journal. we can use the map() function in pandas to achieve this.

• Example code: main_dataframe['Open Access'] = main_dataframe['ISSN'].map(open_access_dict)

main_dataframe with the name of our dataframe and open_access_dict the name of our dictionary containing Open Access information.

Fill missing Open Access information with 'False'

• Some ISSNs may not have a corresponding entry in the Open Access dictionary. In this case, we may assume that these journals are not Open Access. Fill the missing values in the 'Open Access' column with 'False'.

For the sake of easing the next steps we will create a second version of the ERIH-PLUS dataset, filtering out all journals that are not in OC meta. This will be called ERIH-PLUS_filtered. This dataset will also have the OMID in it to be able to connect it with out dataframes that maps issns

Our second and third research question are "what are the disciplines that have more publications? What are countries providing the largest number of publications and journals?" so we need to include in our dataframe information about each journal country and the discipline it belongs to.

These information are both present in the ERIH-PLUS dataset.

Disciplines:

As we can see every journal can have multiple disciplines:

The ERIH-PUS Disciplines are separated with ,

OECD Classifications disciplines are separated with ;

In order to count the disciplines and understand which one has the highest number of publications we will need to disassemble them and map both classifications in order to prevent information loss on the two classifications.

Luckily, we don't have too many disciplines to map, given that ERIH-PLUS has 30 disciplines and OECD has 14.

To create the table we need we will write a python function that takes in input the ERIH-PLUS filtered dataset

Countries:

For the countries we will do the same.

This is relatively less complicated since, from a first exploration, there are not multiple countries in the ERIH-PLUS filtered dataset.

We will take into consideration that some journals have no specified country. To tackle this issue we can try to see if it is present in the DOAJ dataset.

Checking wether missing countries in erih-plus are present in doaj dataset.

DOAJ dataset has the following structure:

where, Country of Publisher contains the same country information as ERIH-PLUS Country of Publication.

We could write a python function that does the following:

1. Selects all ISSN in ERIH-PLUS that have no country
2. checks if the ISSN exists in DOAJ dataset
3. if yes, retrieves the country information in the Country of Publisher column
4. add that information to ERIH-PLUS filtered dataset.

We will merge our two datasets, the disciplines one and the countries one , by means of the unique OMID, finally we will also add a column stating wether the journal is open access or not. Here is how we envision the final dataset to look like:

A	B	C	D	E

Also DOAJ has a "subject" column with a discipline classification. We need to discuss if we want to include that or not.

We now need to count:

1. Number of publication for each SSH journal in OCMeta
2. Total number of SSH Publications in OCMeta

We could do this by means of a sparql query to OpenCitations endpoint

https://opencitations.net/meta/sparql

for each OMID we will ask for the number of publication and we can save this information adding a column in the final dataset.

A	B	C	D	E	F

To find out the coverage of SSH publication we will sum the values in Publication_count column

We still need to find out how to retrieve the total number of publication a Journal has, that has to be found somehow in ERIH-PLUS. Once we have that we can compute a percentage.

To find the countries that have the highest number of publication we will sum the values in Publication_count column for every country.

This information will be saved in a dictionary that has the Country name as key and the total count of publication for each country as value.

To find the disciplines with the highest number of publication we will follow the same process but it will need to be done with both of the disciplines classifications.

To answer to our research question "How many of the SSH journals are available in Open Access according to the data in DOAJ?"

We will simply count the rows that have a "yes" value in the Open Access column.

For each visualization, we use Python libraries like Matplotlib, Seaborn, Plotly to create the charts and maps.

Analyzing the coverage of publications in SSH journals

• Visualization: Bar chart showing the number of publications for each journal in the ERIH-PLUS list. The x-axis represents the journals, and the y-axis represents the number of publications. Identifying the disciplines with the most publications

• Visualization: Bar chart showing the number of publications for each discipline. The x-axis represents the disciplines, and the y-axis represents the number of publications. Identifying the countries providing the largest number of publications and journals

• Visualization 1: Choropleth map showing the number of publications by country. Each country is colored according to the number of publications, with darker colors representing higher publication counts.

• Visualization 2: Bar chart showing the number of journals for each country. The x-axis represents the countries, and the y-axis represents the number of journals. Fetching the list of Open Access journals from DOAJ

• Visualization: Pie chart showing the proportion of Open Access SSH journals (according to ERIH-PLUS) based on DOAJ data. The chart will have two segments: Open Access journals and Non-Open Access journals.