1137_om_not-searchable
/en/arbeid-og-lonn/statistikker/sykefratot/kvartal
1137_om
statistikk
2014-03-20T10:00:00.000Z
Labour market and earnings;Immigration and immigrants;Health
en
false

Sickness absenceQ4 2013

Content

About the statistics

Definitions

Name and topic

Name: Sickness absence
Topic: Labour market and earnings

Next release

Responsible division

Division for Labour Market and Wage Statistics

Definitions of the main concepts and variables

Sickness absence
Absence from work due to one's own illness.

Proportion of employees with sickness absence
The number of employees with doctor-certified sickness absence on a specific reference date in per cent of total number of employees without taking into account of part-time jobs. From 2015 the reference date is Tuesday in the week covering 16th day in the last months of the quarter.

From 2nd quarter 2002 to 4th quarter 2014, the reference date was the Tuesday before the last Wednesday in the Quarter, unless that ended up being the Easter- or Christmas week. In these situations, the reference date was set a week earlier.

Employees with sickness absence adjusted for partial sick leave
The number of employees with doctor-certified sickness absence on a specific reference date adjusted for partial sick leave. This means that a partial sick leave of 80 per cent counts as 0.8 and not as one. From 2015 the reference date is Tuesday in the week covering 16th day in the last months of the quarter. We also compute the rate of this figure by dividing by total number of employees without taking into account of part-time jobs.

Sickness absence rate
The sickness absence rate measures man-days lost due to own sickness as a percentage of contractual man-days.

Sickness absence rate = {sickness absence man-days*100} / contractual man-days

Man-day: one man-day corresponds to the length in time of one working day for a person in a full-time position (100% position).

Sickness absence man-day: a sickness absence man-day corresponds to one sickness absence day for a person in a full-time position (100%) and who is on full sick leave (100% degree of disability).

Sickness absence man-days in the period = {sickness absence days in the period} x {position proportion} x {degree of disability}.

Sickness absence day: a day during which one normally is supposed to be at work but is not because of one's own illness. The number of sickness absence days in a case of sickness absence is equal to the number of calendar days within the statistics period to which the case of sickness absence pertains minus any weekends and holidays.

Position proportion: Position proportion is reported from the employers to the A Scheme Register. The proportion is allowed to take values between 0 and 1. Until 2014 position proportion was calculated as contractual working hours, divided by 37.5. Hence, we assumed 37.5 to be working hours of a full-time position, which was a simplification. From 2015 a full-time position will have position proportion = 1, regardless of the number of hours per week.

Degree of disability: whether the person in question is on full or partial sick leave, and to what degree, is indicated by the degree of disability. The degree of disability is between 20 percent and 100 percent, where 100 means full sick leave. If a sickness absence case consists of more than one medical certificate, we use the average of the degrees of disability for the medical certificates in question.

Contractual man-days in a period = {contractual working days in the period} x {position proportion} x {vacation correction factor}

Working day: a day during which one normally is supposed to be at work.

Contractual working days: the number of working days that a person has agreed on with his or her employer to work in a period. We define possible working days as the number of calendar days that the employee relationship is lasting minus weekends and general holidays. For many employees such a generalisation will not be correct. At the aggregate level, and when looking at a given period, we believe nonetheless that this approach works reasonably well. For reasons of quality, in the statistics we established a ceiling for how many employee relationships a person may have simultaneously.

Vacation correction factor: Average vacation absence in the quarter distributed by sex and industry is calculated from the Labour Force Survey and is converted in the sickness absence statistics to a vacation correction factor that varies between 0 and 1. 

Contractual working hours:
Contractual working hours for each employee is reported in hours per week with two decimals to the A Scheme register. This number is being used to calculate position proportion, if this proportion is missing. Earlier, all the job relations in the population were divided into six groups formed by sex and the interval values of scheduled working hours from the Employee Register (4-19 hours per week, 20-29 hours per week, and 30 hours per week and above). For each job relation in a group, the scheduled working hours was estimated as the average scheduled working hours in this group according to the Labour Force Survey (LFS).

Case of sickness absence: a case of sickness absence is defined by a common personal identification number, a common start date for the medical certificate date, and a common organisation number for the company. A case of sickness absence will consist of the initial medical certificate and also medical certificates being extensions of the initial one. If an employee on sick leave has more than one job relation, we assume that the person has sick leave from all job relations. For example, a person having two job relations being active during two medical certificates, and these medical certificates are separated by some time where the person is not on sick leave, the person will have four cases of sickness absence.

Counting employee relationship: in some instances, it is necessary to select one employee relationship as counting for persons who are registered with more than one employee relationships simultaneously at a given reference point or in a given period. For instance we have to do this if we wish to divide employees by industry and municipality of the workplace.

The counting employee relationship is selected according to the following criteria, which are the same criteria used by Statistics Norway in its Register statistics of employees:

The employee relationship having the longest contracted working hours per week is the counting employee relationship of a person. In the case of equal working hours, the one with the latest start date is chosen.

Age: Age is defined as the age per the 16th of each month. Before 2015, age was defined as the age at the end of the statistics period.

Duration: Duration is measured among ongoing sickness absences at the reference point, which causes the duration to be shorter than if we were looking at terminated sickness absences.

Standard classifications

Industry is coded according to the Standard Industrial Classification, SIC2007 .

Education is coded according to the Norwegian Standard Classification of Education, NUS (NOS C 751) .

Occupation is coded according to the Norwegian Standard Classification of Occupations (NOS C 521)

Sector: The classification is based on institutional sector codes from the Central Coordinating Register for Legal Entities. The following classification is used in the tables:

Central government administration

County administration

Municipal administration

Private sector (incl. public enterprises)

Administrative information

Regional level

National, county, and municipality level.

Frequency and timeliness

The statistics are published quarterly and annually.

International reporting

Not relevant

Microdata

Not relevant

Background

Background and purpose

The statistics are intended to shed light on sickness absence trends in the aggregate and distributed by industry, sector, hours worked and various personal variables connected with those on sick leave. These are intended to form the basis for developing and evaluating measures aimed at sickness absence ('IA-agreement'). Publication began in 2001 and replaced the partially abridged statistics that had covered the central government and municipal sectors and parts of the private sector

Users and applications

Major users of the statistics are the authorities, employers' and employees' organisations, individual companies and researchers.

Aims that have governed the planning of the statistics are:

1. The possibility for employers and funds-appropriating authorities to make reliable estimates of expenses.

2. The possibility for employers' and employees' organisations to evaluate the efficiency of various measures implemented to reduce sickness absence.

3. The possibility of identifying risk groups in order to implement preventive measures.

4. The possibility of comparison over time as the basis for research in the area.

Admission to the figures before release date
As part of the collaborative effort on sickness absence with Statistics Norway (SSB) and The Norwegian Labour and Welfare Administration (NAV), NAV receives figures before release date on both self-certified and doctor certified sickness absence. SSB and NAV then releases the figures simultaneously.

Equal treatment of users

Not relevant

Coherence with other statistics

Many different bodies produce or have produced sickness absence statistics that each covers various sectors of business and industry. The Confederation of Norwegian Business and Industry (NHO) compiled earlier sickness absence statistics covering blue-collar and white-collar workers in mining, manufacturing and general contracting activities. Now NHO has stopped their data collection and receives data from Statistics Norway in stead.

The Norwegian Association of Local and Regional Authorities (KS) compiles sickness absence statistics for permanent employees in nearly all municipalities and county authorities as well as for employees of companies that are associate members of KS. The Ministry of Labour and Government Administration compiles sickness absence statistics for all employees for whom the central government is the employer. A more detailed comparison of the various measurement figures for sickness absence has been made in the report from the Central Statistics on Sickness Absence pilot project. Other absence statistics are also available. For the retail and financial intermediation industries, the Federation of Norwegian Commercial and Service Enterprises and the Norwegian Financial Services Association (FNH) compile statistics respectively.

The Labour Force Survey, which Statistics Norway publishes quarterly, also contains some information about sickness absence. The target figure used in this survey is the number of employees who have been absent during the whole registration week in per cent of employees in total. The figure is given by gender and group of industry. This target-figure is less comparable with those in the other Norwegian statistics. The Central statistics of sickness absence for instance, include persons on partial sick leave and also cases of sickness absence shorter than one week. In addition these statistics also take into account both the working time and the duration of the sickness absence when the portion of sickness absence is calculated.

Legal authority

Statistics Act §3-2 (administrative registers)

Statistics Act §2-2 (obligation to provide information)

Statistics Act §2-3 (compulsory fines)

EEA reference

Not relevant

Production

Population

The statistics does not include self-employed or conscripts. Further more it does not include absence related to a child's illness or family and maternity leave.

Doctor-certificated sickness absence
The population is all employees aged 16-69 residing in Norway and registered in the A Scheme (a-ordningen) Register classified as ordinary and maritime jobs during the reference week. The Reference week is the weeks each month containing the 16th. During a quarter there are Three Reference weeks, and the employment has to be active in at least one of these weeks to be counted.

Under the A Scheme Act, the reporting duty an employer occurs from 2015 for all his employees with payroll or expenditure allowance above 1000 Norwegian kroner per annum. Before 2015 the Employee register population was limited to employees having a job relation scheduled to more than four hours a week and lasting more than six days. Self-employed are not included in the statistics.

Self-certificated sickness absence
The population for the survey is all businesses operating in Norway with a few exceptions. Because of difficulties in obtaining reliable information about self-certificated sickness absence, the following businesses are kept outside the survey: private households with employed persons, extra territorial organizations, investment companies, extraction of crude petroleum and Natural gas, water transport and animal husbandry service activities (except veterinary activities).

Data sources and sampling

Doctor-certified sickness absence

The Sick Leave Register is the main source of information for doctor-certified sickness absence. The Norwegian Labour and Welfare Administration (NAV) is responsible for the register.

The Sick Leave Register is based on the local NAV offices registrations of medical certificates ("Sykemeldingsattest IA") as part of its sickness benefits routine. These registrations covers all doctor-certified absences due to a person's own illness. The Sick Leaves are linked to employees by the national identity number.

As from January 2015 the Employee register was replaced by a new register on monthly reporting of employee and payroll information (a-melding) to NAV, Tax Administration and Statistics Norway. Jobs classified as ordinary and maritime in the A Scheme register corresponds to the population of employees in the earlier Employee register. Still, the earlier Employee register was supposed to contain all job relation scheduled to more than four hours a week and lasting more than six days. Under the A Scheme Act the reporting duty for an employer occurs for all his employees with payroll or expenditure allowance above 1000 Norwegian kroner per annum. Due to this new and more strict reporting duty, the A Scheme register have a wider scope than the earlier Employee register, and this has produced a minor level shift from the 1st quarter of 2015.

Self-certified sickness absence

The statistics on self-certified sickness absence are based on data reported by a sample of establishments. A random sample of nearly 10 000 establishments is drawn from The Central Register of Establishments and Enterprises (CRE). The establishments are stratified by industry, size and county of work. All the establishments in the population with at least 5 employees are divided into 4 groups depending on the number of employees. The establishments are stratified on most Divisions (2-digit groups) in the Standard Industrial Classification (SIC2007). This ensures a representative sample in 44 groups of industry, sizes of establishment and counties. All establishments in the same strata have the same probability to be drawn. The probability to be drawn increases by size. None of the small establishments (four or less employees) are drawn, while all big establishments (more than about 150 employees) are included. This ensures that the survey covers about 35 per cent of all employees while only 5 per cent of all establishments are surveyed. While the biggest establishments are always surveyed, most others participate for four years at a time. Therefore, we replace almost a quarter of the sample each year. The drawing of the sample is coordinated with several other surveys in Statistics Norway in order to disperse the burden placed on smaller and medium sized establishments.

Collection of data, editing and estimations

The questionnaires are sent out the last week of each quarter to the enterprises having establishment(s) in the sample. The personnel office of the enterprise fills in the questionnaire or forwards it to the personnel office of their establishment. The deadline for delivery is about four weeks after receiving the questionnaire. An establishment may contact the Division for Data collection at Statistics Norway if it is impossible to separate the sickness absence figures for the chosen establishment. In this case the enterprise may give figures for the enterprise as a whole.

Doctor-certified sickness absence
The Norwegian Labour and Welfare Administration performs a number of controls and corrections on the doctor's certificates especially related to overlapping/inconsistencies in the dates.

Employees from the A Scheme Register
There are a number of controls in the reporting system. In addition, Statistics Norway makes a number of checks and reworking to arrive at the active employments.

For employments with position proportion over 1.2, the position proportion is set to 1.2. The total postion proportion for a person is not allowed to exceed 1.6. For persons where this is the case, some of the employments are removed, because we do not believe them all to be active in the reference week.

Until 2014, there was a limit regarding how many employments one person could have.

Self-certified sickness absence
Absolute checks: verification controls in the last stage of the optical data registration to reveal obvious errors from the optical reading, misunderstandings or inaccuracy in the reporting. An example would be wrong totals compared to the partial figures.

Evaluation checks: these checks aim to reveal errors due to extreme values reported or due to incorrect number of employees reported. For this purpose a check variable is being used, sickness absence cases per 100 employees. Large establishments that report extreme values on this variable will be more closely examined. Data from whole enterprises will also be adjusted in proportion to the portion of employees the sampled establishments represent in the enterprise as a whole. Before the final estimation, we also check that the net samples in each industry and size strata are large enough. If the net samples are too small for some strata, they will be added to the "neighbouring" strata in the same industry.

Doctor-certified absence
Persons with a so-called active self-certificate are included in the statistics as ordinary doctor-certified sickness absence.

Measures of sickness absence
The sickness absence rate is sickness absence man-days in per cent of contractual man-days. The proportion of employees with sickness absence at a given date is also calculated. See under 'Definitions' for more information regarding these measures. 

Self-certified absence
Sheltered workshops and establishments in the following industries are not a part of the sample: Animal husbandry service activities (except veterinary activities), extraction of crude petroleum and natural gas, water transport, private households with employed persons and extra territorial organizations. In the estimation, employees from these groups are represented by employees from other industries.

Estimation at national level
A model-based ratio estimator is used to inflate sample data of self-certified sickness absence to population level. The ratio model corresponds to a linear regression model without a constant term and with the disturbance term that are proportional with the number of male (female) employees. Auxiliary variables about total number of male (female) employees in each establishment in the population are used. The figures are taken from the A Scheme Register. The dependent variable is the number of self-certified sickness absence man-days for male (female). A weighted least square method is used to fit the regression models. The population totals for each group are predicted by using the groups model estimates and the population sums of the number of employees in each group.

Because the smallest establishments are not sampled, a "Cut-off"-estimator is being used for these strata. This means that the number of male (female) employees in the population of the smallest strata is being multiplied by the ratios for the second smallest strata. This method is being done separately for each industry strata. This method assumes that the ratios in the strata with the smallest establishments are similar to the corresponding ratios in the strata with the second smallest establishments. If the ratios are very different we introduce a bias.

Estimation at county level and institutional sector level
Self-certified sickness absence by county of work and institutional sector is estimated with a different method than the national level estimates, because the sample is not stratified on these variables. For the figures by county and sector, we use model based ratio estimators for different groups.

The estimations of sickness absence by county are done separately for two industry groups, sex and the 19 counties of work. One industry group consists of agriculture, forestry and fishing, and industrial activities; the other group is service activities.

In the estimation of figures by institutional sector, each of the tree sectors 1) central government, 2) counties and municipalities, and 3) private sector and government enterprises are estimated separately for male and for female for groups of industry. Depending on what the sample sizes allow, different groups of industry are used for each of the tree sectors.

The different model-based ratio estimators can add up to figures of totals that are a little different from the official national totals from the design-based ratio estimator. Due to that, we use the county and sector divided figures to make percentage distributions, which the official national totals are broken down with, so that the totals also are right.

Seasonal adjustment

Seasonally adjusted data are calculated by using the X12-ARIMA software. The adjustment is done indirectly. We separately seasonally adjust the numerator and denominator for gender divided sickness absence rates certified by doctor and for self-certified sickness absence rates by gender. We let the X12-ARIMA automatically decide on which seasonal adjustment decomposition, forecast model, seasonal filter and trend filter that gives best results for each time series. These optimal choices are locked between yearly reviews of the method. Regression models in X-12-ARIMA pre-adjust the series, where we specify variables for influenza irregularities (numerators only), the number of working days in the quarter and differentiated proportions of absence on holiday (denominator only). All these pre-adjustment variables are deseasonalized. A level shift is also specified for sickness absence man-days certified by a doctor from third quarter 2004 (doctor certification reform), and a additive outlier third quarter 2009 is specified for self-certified sickness absence man-days for men (beginning of the swine influenza). Doctor-certificated sicknes absence and contractual man-days are pre-adjusted for a level shift in the 1st quarter of 2015 because of the transition to the A Scheme register, and for an Extreme value bacause of starting problems in the 1st quarter of 2015. Self-certificated sickness absence is also pre-adjusted for the startup of the A Scheme register, specified as a temporary change from the 1st quarter of 2015. The reason for this pre-adjustment of self-certificated sickness absence is that information regarding the number of employees from the A Scheme register is being used in the estimation.

For the seasonal- and influenza adjusted figures, the estimated effect of the extreme value linked to the startup of the A Scheme in the 1st quarter of 2015 is corrected away from the time series for doctor-certificated sickness absence and contractual man-days. In this way the extreme value is not visible.

For more detailed information about the seasonal adjustment of this statistics, please see  About seasonal adjustments .

Confidentiality

Not relevant

Comparability over time and space

In the autumn of 2001 the authorities and the organisations of employers and employees signed an agreement aiming at reducing the level of sickness absence. One of the means was to offer the employees an expanded period of self-certified sickness absence beyond three days combined with dialogues with the employer. This effort might turn some of the sickness absence from the doctor- to the self - certified kind. Due to this - the survey has to capture a greater part of the sickness absence than earlier, and our questionnaire for the 4th quarter of 2001 was changed accordingly. This change implies more questionnaire-based reporting from the establishments at the expense of register data based on the doctors' reports.

The effect of the A Scheme Register on the sickness absence rate
Introduction of the A Scheme register to replace the Register of Employees and Employers (Aa-registeret), from the 1st quarter of 2015, gives somewhat changing figures for the number of employees. This is due to a somewhat broader coverage of the group, and a better quality. The transition to the new data has had a small, but still observable effect on the sickness absence rate at the aggregate level. The effect of the transition to the new data is small also when it comes to self-certified and doctor-certified sickness absence separately. But since the self-certificated absence is lower than the doctor-certificated, this small effect provides a greater impact on the numbers of alteration for self-certified absence than for doctor-certified absence, and for the total absence. To get completely  comparable figures for self-certificated sickness absence, the respective sickness absence rate before 2015 should be reduced by 4 hundredth. To get completely comparable figures for doctor-certificated sickness absence, the respective sickness absence rate before 2015 should be reduced by only 1 hundredth. To get completely comparable figures for the total sickness absence, the total sickness absence before 2015 should be reduced by 5 hundredth. These calcalculations are obtained by taking differences between official seasonal- and influenza adjusted sickness absence rates and the corresponding sickness absence rates that also are corrected for the estimated effect of the level shift in the seasonal adjustment programme. These calculations underlie Statistics Norways's adjusted estimate of percentage change of the sickness absence during the IA-period.
We cannot exclude that there are slightly larger level shifts for some industries, age groups or other detailed numbers. Especially the fact that the A Scheme register captures more of the small job relations, than the Aa-registeret did.

Direct reporting of position proportion is now being used, while Aa-registeret only gave information about contractual working hours, and one assumed that 37.5 hours per week equaled a full position. This may also have affected the figures, especially for industries with a high degree of shift and rotation.   

New industry classification
A new industry classification (SIC2007) was introduced from the publication of the first quarter of 2009. Compared with earlier published figures classified by the previous industry standard (SIC2002) there will be a break in the time series. Here is an article presenting general information about the new Norwegian industry classification .

It is not a one to one link between all the detailed codes of SIC2002 and SIC2007. However 1 st quarter 2009 all active establishments in the Central Register of Establishments and Enterprises are coded both after SIC2002 and SIC2007. This information, linked to the Sick Leave Register and the Employee Register, is utilized to construct conversion matrixes. These conversion matrixes are used to calculate aggregated SIC2007 divided historical figures from aggregated SIC2002 divided figures.

For the variable doctor-certified sickness absence man-days and for contractual man-days the conversion calculation are made separately for males and females in 4 age groups (16 &– 24, 25 &– 39, 40 &– 54 and 55 &– 69 years) for the most detailed industry code as possible. For the conversion of the variable doctor-certified sickness absence man-days the conversion percentage distribution are calculated from doctor-certified sickness absence man-days 1st quarter 2009. For the conversion of the variable contractual man-days the conversion percentage distribution are calculated from contractual man-days 1st quarter 2009.

Self-certified sickness absence man-days figures are converted from SIC2002 to SIC2007 within the industry stratums used in the sampline and estimation before and after the change. This is done separately for males and females based on conversion percentage distributions made from the variable contractual man-days 1 st quarter 2009. Industry conversion of the denominator used in the self-certified sickness absence rate are done separately for males and females within the industry stratums in the same way as the self-certified sickness absence man-days figures are converted.

Gender and SIC2007 divided figures are converted quarterly for the period 2000 &– 2007 under the assumption that the different detailed conversion percentage distributions are constant and equal to the once in 1 st quarter 2009.

Accuracy and reliability

Sources of error and uncertainty

Collection and processing errors (self-certified sickness absence)
Errors may occur in many parts of the data collection process. They may for instance occur when filling in the questionnaire, registration of the data or through incorrect optical reading or revision of data. Different interpretations of the questionnaires' notions may also cause faulty data. Many faults are however revealed and corrected by means of an elaborate process of checks and revision. The questionnaire was revised the 4th quarter 2001. Before the revision we asked for the total sickness absence of 1 - 3 days (both self- and doctor-certified) while we after the revision ask for all the self-certified sickness absence only (independent of duration). This may cause faults if some establishments do not change their reporting routines.

Sampling errors (self-certified sickness absence)
All surveys are bound to have a certain level of sample uncertainty. The uncertainty generally increases with a declining numbers of observations used in the survey. We get a measure of the variance by calculating the standard deviation in per cent of the estimate. The calculation based on the figures from the 4th quarter 2004 reveals a relative standard deviation of 0.8 per cent in the total number of man-days lost because of sickness absence. The relative standard deviation for each of the 12 groups of industry used in our survey varies and has an average of 3.2 per cent.

The population of establishments constantly changes because of new establishments, bankruptcies, fusions and changes in kind of industry (SIC 94). The original sample of 1997 was not changed before the 1st quarter 2002. If new establishments within the same group of industry and size during that period have other averages of sickness absence per employee than the 1997 population, a bias in the estimates will arise. From 2002 the sample is upgraded annually. Since the smallest establishments are held outside the sample, particular uncertainty is attached to the calculated numbers within this group.

In all voluntary questionnaires there will be a certain degree of non-response. With the new sample from the 1st quarter 2002 we reached a 65 per cent response rate after reminders. At the start of the survey we had almost the same level of response, but it decreased gradually to 55 per cent towards 2001. This may cause biased estimates, even though the method of estimation is reliable in relation to changes in the structure of industry group and size. An analysis of the non-response establishments revealed the same level of doctor-certified sickness absence as in the establishments that responded.

Because of the low response rate, the questionnaire became a mandatory survey from the 2nd quarter of 2003. This effort raised the response rate to a much higher level, at about 90 percent in the following quarters.

Self-Certified sickness absence
The number of employed men and women in the establishments found in the Register based on the A Scheme is used as factor of estimation. Some establishments report the numbers of sickness absence based on a larger or smaller group of employees than recorded in the register. This might cause a disproportion between the number of sickness absence reported and the corresponding number of employees. This phenomenon represents a potential source of bias, but occurrences of larger disproportion are revealed and corrected by means of our quality checks.

Doctor-certified sickness absence
A good deal of sick leave is registered that cannot be linked to employee relationships. Compared with all sick leaves, this corresponds to approximately seven percent. Analyses indicate that most by far are sick leaves that do not pertain to employee relationships subject to notification, and they are therefore kept out of the data basis. However, keeping them all outside gives us a certain level of under coverage in the statistics.

The transitions from sickness absence days to man-days and from possible working days to possible man-days went, until 2014, via the calculated variable "position percentage". On the person level we only had information on whether the employment is full-time or short or long part-time. By comparing 37.5 hours per week with what part-time or short or long part-time employees divided by sex responded in the LFS that they had as fixed/contracted working hours per week, we got the "position percentages". This generalisation produced errors for many individuals, but produced fewer errors on the aggregate level.

Reporting of position percentage
During the first quarters of 2015 there have been deficiencies in reporting of position percentage. For a lot of these cases, position percentage is calculated based on the number of paid working hours. A work is going on to improve the quality of the Reporting, while Statistics Norway at the same time tests Methods to adjust for deficiencies in Reporting.

Revision

Not relevant