Projecte

General

Perfil

Accions
Copiar enllaç

Tasca #7313

tancat
FJ FJ

Les peticions de registres a Scopus va DOI són massa breus

Tasca #7313: Les peticions de registres a Scopus va DOI són massa breus

Afegit per Ferran Jorba fa més de 3 anys. Actualitzat fa aproximadament 3 anys.

Estat:
Tancada
Prioritat:
Normal
Assignat a:
Ferran Jorba
Categoria:
Tecnologia
Temàtica prevista:
OAI: recol·leccions
Inici:
05-07-2022
Data de venciment:
12-07-2022
Paraula clau:

Descripció

Des de fa uns dies, quan demanem registres (abstracts, que en diuen ells) a Scopus retornen registres breus, i en general l'error que veieu en el correu és d'aquest tipus:

Traceback (most recent call last):
  File "/home/ddd/bin/scopus2ddd.py", line 500, in <module>
    collections, notes, marcxml=marcxml)
  File "/home/ddd/bin/scopus2ddd.py", line 176, in scopus2marc
    language = scopus['abstracts-retrieval-response']['language']
KeyError: 'language'

Com que he vist que l'error es repeteix, avui he escrit a través del formulari https://service.elsevier.com/app/contact/supporthub/researchproductsapis/ aquest correu (el valor de la nostra API key l'he alterat):

From: Ferran Jorba
Date: 05/07/2022 09.43 AM

Dear Sirs,

we often use the Scopus abstract api to retrieve abstracts from scopus. We use our university (Universitat Autònoma de Barcelona) API key. I use Python using Requests with a simple call like this one:

def get_remote_scopus_xml_record_via_doi(doi):
 verbose = False
 headers = {
 'X-ELS-APIKey': '373257c0c78446899b6fghd10b1567125b',
 }
 url = 'https://api.elsevier.com/content/abstract/doi/%s' % (doi)
 if verbose:
  print('#request: %s' % (url), file=sys.stderr)
 request = requests.get(url, headers=headers)
 if request.status_code == 200:
  scopus_xml_record = request.text.replace('><', '>\n <') + '\n'
 else:
  scopus_xml_record = ''
 try:
  print('Error: %s not found in Scopus' % (doi), file=sys.stderr)
 except UnicodeEncodeError:
 pass
return scopus_xml_record

Since a few days, we are getting shorter records. For example, this call:

https://api.elsevier.com/content/abstract/doi/10.5194/tc-15-2803-2021

From the logs I have, on 23-jun-2022, at 09:53:50, I got this response:

<abstracts-retrieval-response xmlns="http://www.elsevier.com/xml/svapi/abstract/dtd&quot; xmlns:dn="http://www.elsevier.com/xml/svapi/abstract/dtd&quot; xmlns:ait="http://www.elsevier.com/xml/ani/ait&quot; xmlns:ce="http://www.elsevier.com/xml/ani/common&quot; xmlns:cto="http://www.elsevier.com/xml/cto/dtd&quot; xmlns:dc="http://purl.org/dc/elements/1.1/&quot; xmlns:prism="http://prismstandard.org/namespaces/basic/2.0/&quot; xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd&quot; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;>
<coredata>
<prism:url>https://api.elsevier.com/content/abstract/scopus_id/85108277599<;/prism:url>
<dc:identifier>SCOPUS_ID:85108277599</dc:identifier>
<eid>2-s2.0-85108277599</eid>
<prism:doi>10.5194/tc-15-2803-2021</prism:doi>
<dc:title>An improved sea ice detection algorithm using MODIS: Application as a new European sea ice extent indicator</dc:title>
<prism:aggregationType>Journal</prism:aggregationType>
<srctype>j</srctype>
<subtype>ar</subtype>
<subtypeDescription>Article</subtypeDescription>
<citedby-count>0</citedby-count>
<prism:publicationName>Cryosphere</prism:publicationName>
<dc:publisher>Copernicus GmbH</dc:publisher>
<source-id>11700154363</source-id>
<prism:issn>19940424 19940416</prism:issn>
<prism:volume>15</prism:volume>
<prism:issueIdentifier>6</prism:issueIdentifier>
<prism:startingPage>2803</prism:startingPage>
<prism:endingPage>2818</prism:endingPage>
<prism:pageRange>2803-2818</prism:pageRange>
<prism:coverDate>2021-06-18</prism:coverDate>
<openaccess>1</openaccess>
<openaccessFlag>true</openaccessFlag>
<dc:creator>
<author seq="1" auid="57221731896">
<ce:initials>J.A.</ce:initials>
<ce:indexed-name>Parera-Portell J.A.</ce:indexed-name>
<ce:surname>Parera-Portell</ce:surname>
<ce:given-name>Joan Antoni</ce:given-name>
<preferred-name>
<ce:initials>J.A.</ce:initials>
<ce:indexed-name>Parera-Portell J.A.</ce:indexed-name>
<ce:surname>Parera-Portell</ce:surname>
<ce:given-name>Joan Antoni</ce:given-name>
</preferred-name>
<author-url>https://api.elsevier.com/content/author/author_id/57221731896<;/author-url>
<affiliation id="60023020" href="https://api.elsevier.com/content/affiliation/affiliation_id/60023020&quot;/>
<affiliation id="60027844" href="https://api.elsevier.com/content/affiliation/affiliation_id/60027844&quot;/>
</author>
</dc:creator>
<dc:description>
<abstract xmlns="" original="y" xml:lang="eng">
<publishercopyright>© 2021 Joan Antoni Parera-Portell et al.</publishercopyright>
<ce:para>The continued loss of sea ice in the Northern Hemisphere due to global warming poses a threat to biota and human activitie\s, evidencing the necessity of efficient sea ice monitoring tools. Aiming at the creation of an improved sea ice extent indicator covering the European regional seas, the new IceMap500 algorithm has been developed to classify sea ice and water at a resolution of 500 m at nadir. IceMap500 features a classification strategy built upon previous MODIS sea ice extent algorithms and a new method to reclassify areas affected by resolution-breaking features inherited from the MODIS cloud mask. This approach results in an enlargement of mapped area, a reduction of potential error sources and a better delineation of the sea ice edge, while still systematically achieving accuracies above 90 %, as obtained by manual validation. Swath maps have been aggregated at a monthly scale to obtain sea ice extent with a method that is sensitive to spatio-temporal variations in the sea ice cover and that can be used as an additional error filter. The resulting dataset, covering the months of maximum and minimum sea ice extent (i.e. March and September) over 2 decades (from 2000 to 2019), demonstrates the algorithm's applicability as a monitoring tool and as an indicator, illustrating the sea ice decline at a regional scale. The European sea regions located in the Arctic, NE Atlantic and Barents seas display clear negative trends in both March (-27.98 ± 6.01 × 103 km2yr-1) and September (-16.47 ± 5.66 × 103 km2yr-1). Such trends indicate that the sea ice cover is shrinking at a rate of ∼ 9 % and ∼ 13 % per decade, respectively, even though the sea ice extent loss is comparatively ∼ 70 % greater in March. </ce:para>
</abstract>
</dc:description>
<link href="https://api.elsevier.com/content/abstract/scopus_id/85108277599&quot; rel="self"/>
<link href="https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85108277599&origin=inward&quot; rel="scopus"/>
<link href="https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85108277599&origin=inward&quot; rel="scopus-citedby\
"/>
</coredata>
<affiliation id="60029942" href="https://api.elsevier.com/content/affiliation/affiliation_id/60029942&quot;>
<affilname>Centre Eau Terre Environnement</affilname>
<affiliation-city>Quebec</affiliation-city>
<affiliation-country>Canada</affiliation-country>
</affiliation>
<affiliation id="60027844" href="https://api.elsevier.com/content/affiliation/affiliation_id/60027844&quot;>
<affilname>Universidad de Granada</affilname>
<affiliation-city>Granada</affiliation-city>
<affiliation-country>Spain</affiliation-country>
</affiliation>
<affiliation id="60023020" href="https://api.elsevier.com/content/affiliation/affiliation_id/60023020&quot;>
<affilname>Universitat Autònoma de Barcelona</affilname>
<affiliation-city>Cerdanyola del Valles</affiliation-city>
<affiliation-country>Spain</affiliation-country>
</affiliation>
<authors>
<author seq="1" auid="57221731896">
<ce:initials>J.A.</ce:initials>
<ce:indexed-name>Parera-Portell J.A.</ce:indexed-name>
<ce:surname>Parera-Portell</ce:surname>
<ce:given-name>Joan Antoni</ce:given-name>
<preferred-name>
[...]

But today I get this shorter one:

<abstracts-retrieval-response xmlns="http://www.elsevier.com/xml/svapi/abstract/dtd&quot; xmlns:dn="http://www.elsevier.com/xml/svapi/abstract/dtd&quot; xmlns:ait="http://www.elsevier.com/xml/ani/ait&quot; xmlns:ce="http://www.elsevier.com/xml/ani/common&quot; xmlns:cto="http://www.elsevier.com/xml/cto/dtd&quot; xmlns:dc="http://purl.org/dc/elements/1.1/&quot; xmlns:prism="http://prismstandard.org/namespaces/basic/2.0/&quot; xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd&quot; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;>
<coredata>
<prism:url>https://api.elsevier.com/content/abstract/scopus_id/85108277599<;/prism:url>
<dc:identifier>SCOPUS_ID:85108277599</dc:identifier>
<eid>2-s2.0-85108277599</eid>
<prism:doi>10.5194/tc-15-2803-2021</prism:doi>
<dc:title>An improved sea ice detection algorithm using MODIS: Application as a new European sea ice extent indicator</dc:title>
<prism:aggregationType>Journal</prism:aggregationType>
<srctype>j</srctype>
<subtype>ar</subtype>
<subtypeDescription>Article</subtypeDescription>
<citedby-count>0</citedby-count>
<prism:publicationName>Cryosphere</prism:publicationName>
<dc:publisher>Copernicus GmbH</dc:publisher>
<source-id>11700154363</source-id>
<prism:issn>19940424 19940416</prism:issn>
<prism:volume>15</prism:volume>
<prism:issueIdentifier>6</prism:issueIdentifier>
<prism:startingPage>2803</prism:startingPage>
<prism:endingPage>2818</prism:endingPage>
<prism:pageRange>2803-2818</prism:pageRange>
<prism:coverDate>2021-06-18</prism:coverDate>
<openaccess>1</openaccess>
<openaccessFlag>true</openaccessFlag>
<dc:creator>
<author seq="1" auid="57221731896">
<ce:initials>J.A.</ce:initials>
<ce:indexed-name>Parera-Portell J.A.</ce:indexed-name>
<ce:surname>Parera-Portell</ce:surname>
<ce:given-name>Joan Antoni</ce:given-name>
<preferred-name>
<ce:initials>J.A.</ce:initials>
<ce:indexed-name>Parera-Portell J.A.</ce:indexed-name>
<ce:surname>Parera-Portell</ce:surname>
<ce:given-name>Joan Antoni</ce:given-name>
</preferred-name>
<author-url>https://api.elsevier.com/content/author/author_id/57221731896<;/author-url>
<affiliation id="60023020" href="https://api.elsevier.com/content/affiliation/affiliation_id/60023020&quot;/>
<affiliation id="60027844" href="https://api.elsevier.com/content/affiliation/affiliation_id/60027844&quot;/>
</author>
</dc:creator>
<link href="https://api.elsevier.com/content/abstract/scopus_id/85108277599&quot; rel="self"/>
<link href="https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85108277599&origin=inward&quot; rel="scopus"/>
<link href="https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85108277599&origin=inward&quot; rel="scopus-citedby"/>
</coredata>
<affiliation>
<affilname>Centre Eau Terre Environnement</affilname>
<affiliation-city>Quebec</affiliation-city>
<affiliation-country>Canada</affiliation-country>
</affiliation>
<affiliation>
<affilname>Universidad de Granada</affilname>
<affiliation-city>Granada</affiliation-city>
<affiliation-country>Spain</affiliation-country>
</affiliation>
<affiliation>
<affilname>Universitat Autònoma de Barcelona</affilname>
<affiliation-city>Cerdanyola del Valles</affiliation-city>
<affiliation-country>Spain</affiliation-country>
</affiliation>
</abstracts-retrieval-response>

I've inspected the returned headers to seee if there is a quote problem or so, but I haven't found anything.

Al records I request now are this short.

Your help is much appreciated.

Ferran Jorba
Administrador del https://ddd.uab.cat
Servei d'Informàtica
Universitat Autònoma de Barcelona

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #1

  • Estat ha canviat de Creada a En curs
  • Prioritat ha canviat de Normal a Alta

He rebut aquesta reposta automàtica:

Dear Ferran Jorba,

Thank you for contacting the Research Products APIs Support Team. This is an automated acknowledgement to confirm that we have received your email. We will reply as soon as possible, usually within 2-3 business days.

For all future correspondence, please include your unique reference number provided in the subject header of this email. This ensures that we are fully equipped with all necessary information to assist with your query. In the meantime, you can learn more about the Research Products APIs by visiting our Elsevier Developer Portal: https://dev.elsevier.com/.

Kind regards,

The Research Products APIs Support Team

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #2

Dear Ferran Jorba,

Thank you for contacting the Research Products APIs team about the responses from the Abstract Retrieval API.

Upon checking on our system, it appears that the requests that are coming from your API that has less response is not being authorized coming from your institution's IP network.

Can you please confirm if you are submitting the API request from within your institution's direct IP network and still gets the short response?

I will look forward to your reply.

Kind regards,

Jun Bautista
Customer Experience Champion, Research Products APIs
ELSEVIER | Research Products Division
Visit Developer Portal for Research Products APIs

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #3

From: Ferran Jorba
Date: Tuesday, July 05, 2022 01:58 PM GMT

Dear Sirs,

thanks for your prompt reply.

I'm making the request from the same server I have been using for the last years (158.109.2.47), that belong to our univerisity IP range (158.109.x.y).

It is true, however, that I have heard that our end user access policy has changed, in the sense that they have to do a personal login to our university validation system. There are, however, a few fixed IP addresses for walk-in users in the libraries.
What I don't know if this change from our side required a change on Elsevier side (I have not involved in it), and, if so, this can be the cause for this situation, in the sense that you don't recognize our IP address as UAB.

Could it be the cause? What should we do in order to get back the full response for the API?

Thanks,

Ferran Jorba

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #4

Dear Ferran Jorba,

Thank you for your response.

I have checked the IP 158.109.2.47 and I have verified that it is not registered under the Autonomous University of Barcelona, this causes the issue with the API access. Please check with your institution's E-resources network administrator if there has been a recent changes on the IPs of your institution for authentication.

If the IP access is not feasible for you, I can issue a special token for use with your APIKey that will allow you access.

-What specific metadata are you looking to use?

-Are you only looking at data for your institution or will it include other institutions?

-How do you plan to manipulate it for your research? Please provide more details on the relationships you plan on exploring.

-What is the output of your research? Is it a paper or is it a web application?

I will look forward to your reply.

Kind regards,

Jun Bautista
Customer Experience Champion, Research Products APIs
ELSEVIER | Research Products Division
Visit Developer Portal for Research Products APIs

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #5

Dear Sirs,

I see. I'll check with the team that has dealt with our recent condition changes, so we can learn if this has caused this long-used IP be no longer recognized by you.

I'll be back when I have the answer.

Best regards,

Ferran Jorba

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #6

L'Anna Florensa i jo creiem que, efectivament, es deu al canvi de condicions d'accés que la UAB va pactar amb Elsevier, que ara són tots logins personalitzats.

Comptem que afegint-hi les adreces dels servidors com a fixes, com les de walk-in users, es pugui solucionar.

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #7

Per indicació de l'Anna Florensa, la Rosa Gil tramitarà les altes de les adreces IP de Mompou, Taltabull i Moraleda a través de la FECYT.

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #8

L'Anna Florensa ha demanat a la Rosa Gil que sol·licités a la FECYT, que és qui ho gestiona, que ens donés d'alta les adreces IP dels nostres servidors, i ara ja torna a funcionar. De moment s'han donat d'alta:

  • Mompou
  • Taltabull
  • Moraleda (futur substitut de Taltabull)

FJ Actualitzat per Ferran Jorba fa més de 3 anys Accions
Copiar enllaç
 #9

  • Estat ha canviat de En curs a Tancada

CA Actualitzat per Cristina Azorin fa més de 3 anys Accions
Copiar enllaç
 #10

  • Data de venciment s'ha establert a 12-07-2022
  • Paraula clau s'ha establert a NCR

CA Actualitzat per Cristina Azorin fa aproximadament 3 anys Accions
Copiar enllaç
 #11

  • Prioritat ha canviat de Alta a Normal

CA Actualitzat per Cristina Azorin fa aproximadament 3 anys Accions
Copiar enllaç
 #12

  • Paraula clau s'ha suprimit (NCR)
Accions
Copiar enllaç

També disponible a: PDF Atom