Reflections on responses from service providers

The questions answered by the 6 service providers were composed in order to examine the interoperability
principles discussed in the sub-sections of EOSC Interoperability framework. With few exceptions, the intentions of our questions seem to have been clearly understood by the service providers. However, a few questions were unclear when applied in practice, leading to unexpected responses or misinterpretations.

Should end-user behavior be in scope?

A couple of our questions identified some misinterpretation of the scope of our analysis. The service
providers seemed unable or unwilling to distinguish between compliance issues that were directly related
to the features of the service and actual outcomes that were due to user behaviour. This was the case in
several responses when examining the use of proprietary software and data formats. The actual use of open
software and formats may for instance be greater than our findings indicate. A service provider may have
developed a solution fully based on open software and formats and end up replying “no” when asked about
these characteristics. For instance users of the service may in some cases store and make use of Microsoft
Excel spreadsheets when using the service. This is a proprietary and licenced program with proprietary file
formats.
The impact of such potential misunderstandings is expected to be limited to just a few responses.
However, we cannot be sure if the service provider is aware of the extent of end-users who likely practice
non-compliant behaviour, when in the data lifecycle such non-compliant behaviour is most likely to occur or
whether particular groups of end-users are more likely to exhibit non-compliant behaviour. End users may
not be aware that they are practicing non-compliant behaviour. They may not also be aware of available
alternative and compliant methods or tools or may not care if they are compliant to any IF
recommendations.
But before any of these kinds of questions can be addressed, the service providers will most likely have to
decide whether end user behaviour is even something they wish to have in scope. If not, then the
identification root causes of non-compliant end-user behaviour and any recommendations on compliant
behaviour will have to be produced and governed by someone else.

Do one-size-fits-all questions apply?

The six services represent three categories of services, with varying mixes of service portfolio; HPC, data
repositories, and aggregation services. The individual services were naturally only able to reflect on the
interoperability issues that their particular service may have been exposed to. The fact that we have a
slightly disjunct set of respondents limits the conclusions that may be drawn from the survey. In particular,
it is not strictly possible to create a "single number measurement" on the compliance across the complete
set of services, which would have been desirable in order to give an easy assessment on the level of
compliance to the reader.
For example, many of our specific questions may have had limited or no impact on some services. In other
cases, the issues were assessed to be outside of the realm of control of the service provider which is why
the respondent was able to answer “not applicable” or “no” to our questions. Fortunately, many of the
service providers expanded on their negative replies and stated the reasoning for this in their comments.
Unfortunately, the analysis may show a low degree of compliance for some services even when they are
mostly compliant on all the issues that currently have relevance or impact on the service.

However, although this does make the single-number measure aspiration unattainable, this does not make
the common measurements irrelevant across these service categories. The upside of working with a model
that encompasses many or all service types seems to outweigh the downside due to the lack of tailored
approaches to the service categories.

Will future user requirements increase the importance of interoperability frameworks?

Awareness of all the interoperability issues may currently be out of scope for some of the service providers.
Some interoperability issues may be less relevant today but have a long-term impact on the future of the
service.
The highest degrees of compliance amongst the services may be due to careful planning and governance by
the particular service provider or the need of the particular service provider to accommodate advanced
requirements by end users. The maturity of the service may also be influential. If a service has been in
operation for many years and has had deep interaction with its end-users, the more exposure it will likely
have had to interoperability issues.
Some services may also already overlap the categories due to advanced end-user requirements. This could
be the case if a HPC service offers repository-like services for short-time storage or a repository offers users
access to limited compute services. If future development of the services take a user-centric approach, we
can expect even more overlaps across categories.