Scholarly article on topic 'Effectiveness of regulatory interventions on firm behavior: a randomized field experiment with e-commerce firms'

Effectiveness of regulatory interventions on firm behavior: a randomized field experiment with e-commerce firms Academic research paper on "Economics and business"

Share paper
Academic journal
Small Bus Econ
OECD Field of science

Academic research paper on topic "Effectiveness of regulatory interventions on firm behavior: a randomized field experiment with e-commerce firms"

Small Bus Econ (2015) 45:825-840 DOI 10.1007/s11187-015-9666-9


Effectiveness of regulatory interventions on firm behavior: a randomized field experiment with e-commerce firms

Eelko Huizingh . Machiel Mulder©

Accepted: 18 June 2015/Published online: 2 July 2015

© The Author(s) 2015. This article is published with open access at

Abstract Economic regulators use various instruments to stimulate SMEs to change their behavior, but limited evidence exists on the effectiveness of such interventions. We analyze the effectiveness of three ways of providing e-commerce firms guidance about the legal rules on information disclosure. The initial non-compliance is considerable. In two interventions, firm-specific guidance is given by post mail and by e-mail, respectively; in another intervention, guidance on industry level is given by means of dedicated publications and presentations. In all cases, the guidance was given on behalf of the regulatory authority. The effectiveness of the firm-specific guidance is measured through a randomized field experiment, while for the industry guidance a panel analysis is conducted. We find that sending the firms a letter by post mail slightly improved firm compliance, but overall the various forms of guidance appear to be ineffective. Although information-related interventions are commonly used in practice, such regulatory interventions can fail to influence the behavior of SMEs.

E. Huizingh • M. Mulder

Department of Economics and Business, University of Groningen, Groningen, The Netherlands e-mail:

M. Mulder (&)

Authority for Consumers and Markets, P.O. Box 16326, 2500 BH The Hague, The Netherlands e-mail:

Keywords E-commerce • Regulation • Consumer protection • Effectiveness • Field experiment

JEL Classifications D21 • D83 • K20 • M38 •

1 Introduction

Assessing the effectiveness of regulatory intervention becomes increasingly relevant in the public debate on regulation of economic activities. The liberalization of markets over the past decades has relaxed regulatory constraints on the one hand (Wolfl et al. 2009), but has also resulted in more regulatory overview and enforcement of regulatory rules meant to correct market failures resulting from market power, externalities and information asymmetries. This proliferation of regulatory intervention has led to a growing need for information about its effectiveness and efficiency (Niels and Van Dijk 2008). In particular, concerns about the costs of regulation and the risk of inappropriate measures taken by regulators call for a continuing assessment of how effective regulators operate. Prominent examples of such assessments are the policy evaluation studies conducted by the OECD. These studies analyze the impact of regulatory measures on the macroeconomic performance of countries (see, e.g., Nicoletti and Pryor 2006). Information on the effectiveness of regulation can also be helpful in the debate on the legitimacy of regulatory intervention

(Don et al. 2008). Another objective of regulation-effectiveness studies is to increase our understanding of effective strategies to reach regulatory goals. Stafford (2003), for instance, analyzes the effectiveness of different types of hazardous waste regulation and finds that mandatory pollution prevention programs do not affect compliance. Other examples of this type of studies are De Witte and Saal (2010) who find that a light form of price regulation of water utilities is able to foster productivity in the drinking-water industry and Nosenzo et al. (2013) who find that fines are effective in deterring non-compliance with rules, while the effect of bonuses on compliance is weak. On the topic of rule compliance, Telle (2013) concludes that frequent auditing of firms raises their compliance with environmental regulations. In an earlier study, Telle (2009) finds that just the threat of inspection, measured by the predicted probability of inspections, reduces the probability of violation of environmental rules.

Compared to the above interventions, information-related interventions constitute a mild form of regulatory intervention, as they aim to change agent behavior by informing agents about the existence and contents of regulation. Limited previous research suggests such interventions to be promising. For example, Apesteguia et al. (2013) study rules compliance by individuals and analyzes the effects of sending e-mail messages to visitors of libraries based on the speed by which they return items to the library. They find that the e-mail messages help to promote rule compliance, but the authors cannot exclude an alternative explanation, which is that the e-mail messages act as signals that the library cares about rule compliance which triggered the positive response. Iyer et al. (2010), in a study on regulatory compliance by firms, find that informing businesses by sending a letter about the enforcement strategies affects compliance. Communicating higher levels of detection significantly raises compliance with the tax rules. Both studies on information-related interventions suggest that such interventions can be effective, and as they are mild, relatively easy to implement and not expensive to execute regulatory authorities may consider them attractive means to stimulate regulatory compliance. In practice, regulatory authorities commonly use information-related interventions.

To further increase our understanding of the effectiveness of information-related interventions,

we focus on three kinds of such interventions: two of them focus on informing individual firms while the remaining intervention aims at informing the entire industry. Our research builds on Apesteguia et al. (2013) but is directed at firms instead of individuals, which enables us to test whether their conclusions can be extended to SMEs. It complements Iyer et al. (2010), who also study firms, because we focus on the different ways of distributing information on legal rules instead of different levels of information on enforcement strategies.

In this paper, we determine the effectiveness of three regulatory interventions that intend to stimulate compliance behavior of SMEs by increasing their awareness of legal obligations. More specifically, we analyze to what extent different regulatory interventions aimed at e-commerce firms by the Dutch Consumer Authority1 are effective measures to enforce legal obligations on information disclosure. According to Dutch consumer law, e-commerce firms are obliged to present various types of information that are viewed to be useful for consumers in making decisions. Since e-commerce firms tend to be small (over 80 % in our sample is a sole-trader business, see ''Appendix 1'') and young (about 50 % is below 3 years), while the rules on information disclosure are relatively new as well, one may expect that these firms are not fully aware of the legal obligations. Hence, informing these firms about the legal rules may raise compliance. As these firms may respond differently to different ways by which the Consumer Authority informs them about the legal rules, we assess the effectiveness of three different interventions. Two interventions are at the firm level, while one is at the group level. Note that implementing the legal rules does not require much effort by the online shop, as the rules only refer to mentioning specific information on specific places on the websites.

The effectiveness of regulatory interventions can be measured by a number of methods. Bergman (2008) distinguishes qualitative studies, such as peer reviews, customer-satisfaction surveys and event studies, and

1 During the period of research, the Consumer Authority merged with the Netherlands Competition Authority (NMa) and the Independent Regulator of Telecom and Post (OPTA) into the Netherlands Authority for Consumers and Markets (ACM). For reasons of convenience, we only speak of the Consumer Authority in this paper.

quantitative studies, such as econometric studies on microlevel or macrolevel. Each of these studies consists of an ex post analysis, estimating the effect of past interventions on past events. A critical component of such studies is the definition of the counterfactual development; i.e., the development when the regulatory intervention would not have occurred. One way to account for counterfactual development is to conduct studies in which the circumstances affecting firm behavior are controlled. Experiments, both laboratory and field experiments, are therefore becoming more popular in economic research as they enable researchers to assess, for instance, the impact of a specific regulatory measure on firm behavior, holding all other factors equal (Levitt and List 2009).

In a laboratory experiment, Nosenzo et al. (2013) analyze the impact of fines as well as bonuses on compliance. Laboratory experiments can be helpful in analyzing the impact of specific interventions, but the results of such an experiment cannot directly be translated into daily policy because of the constructed character of the experiment. An advantage of field experiments compared to laboratory experiments is that the former combines the advantages of a controlled laboratory experiment with realism as it works with real-life agents not knowing that they are subject to economic analysis. Field experiments are increasingly used to assess the impact of regulatory interventions. Collins (2013) analyzed the impact of a mandatory financial education program on financial behavior of low-income families which were randomly assigned to a treatment group or a control group. Iyer et al. (2010) applied randomized field experiments to assess the effects of different enforcement strategies on compliance with tax rules. Another example is Apesteguia et al. (2013), who used randomized field experiments to analyze the effects of sending e-mail messages to visitors of libraries on the speed by which they return the items to the library.

Following recent research (e.g., Levitt and List 2009; Iyer et al. 2010; Apesteguia et al. 2013; Collins 2013), we conduct randomized field experiments involving consecutive measurements in which we monitor the compliance with legal obligations by online shops. Using data from the Dutch Chamber of Commerce, we first define two representative samples which are used as a treatment group and a control group. In the first firm-specific intervention (Firm

Letter Guidance), each firm in the treatment group receives information on the legal obligations through a personalized2 physical letter sent by the Consumer Authority, while the firms in the control group do not receive such information (see ''Appendix 2''). Before and after this intervention, we monitor the compliance of both groups of firms. In the second firm-specific intervention (Firm Email Guidance), a personalized e-mail was sent to all firms in the treatment group similar to the letter in the first intervention (see ''Appendix 3''). This e-mail was sent about 1.5 year after the firms had received the letter by post mail. Again, we monitor the compliance behavior of both groups before and after the intervention. This e-mail also includes a hyperlink to the website of the Consumer Authority which gives more detailed information on how online shops should look like. The Industry Guidance concerns the Consumer Authority providing information on the legal obligations to all firms in the industry by means of various dedicated publications and presentations in public media. We monitor the compliance of all firms before and a few months after the Industry Guidance was given.

The advantages of our research design include avoidance of non-compliance and self-report bias (Collins 2013), since we observe whether information is disclosed on a firm's website. The observations are unobtrusive as the online shops are not informed about the measurements. The design does suffer from attrition, which is common in longitudinal research, as several websites disappeared during our study period or ceased to be an online shop. The data are collected by visiting each website to determine the availability of information elements by applying a detailed data-collection instrument (see Sect. 3.3).

At the baseline, about 10-25 % of the online shops do not provide the legally required information on firm characteristics, transaction costs and the order process. Regarding two other aspects, the initial compliance of the online shops is even much worse: 60-70 % do not provide the necessary information regarding the time window to reconsider a purchase and the payment

2 By "personalized" we mean that the letters and e-mails were addressed to specific firms including their names in the headings. The letters and e-mails did not include firm-specific information on the performance of the firms regarding the legal rules.

process. These observations confirm our expectation mentioned above that the compliance of e-commerce firms with legal rules on information disclosure is not high. With respect to the regulatory interventions, however, we find that none of the interventions has a significant effect on firm behavior, neither in the short term nor in the long term. During the study period, firm compliance with the legal rules on information disclosure changed hardly, no matter which intervention was used by the Consumer Authority. Although information-related interventions are commonly used in practice, our research suggests that such regulatory interventions in the form of providing only guidance to SMEs can fail to influence firm behavior. Our findings underscore the need for economic regulators to design regulatory policies with proven effectiveness and highlight our limited understanding of what makes SMEs comply with legal obligations.

The structure of this paper is as follows. Section 2 discusses the background of the regulation on information disclosure by e-commerce firms. Section 3 describes the field study, including the design of the randomized field experiments, the interventions, the various measurements and the design of the econometric analysis. Section 4 presents the results, which refer to the initial compliance, the effects of the two types of firm-specific guidance and the effects of the Industry Guidance. Section 5 concludes with the policy implications of our study and suggestions for further research.

2 Regulation of e-commerce

Technological innovations in the telecommunication industry have had major effects on consumers and producers. One of these consequences is the rapid growth in business-to-business and business-to-consumer e-commerce. The share of e-commerce in total turnover of industries was approximately 15 % on average in the industrialized countries in 2012 (OECD 2013). E-commerce appears to have a significant impact on economic growth and trade (Terzi 2011). Buying and selling goods and services through the Internet have many advantages. Online shops give buyers access to global markets with a large variety in products, while sellers can use the Internet for enlarging the geographical area of their market (Grandon and Pearson 2004). Hence, e-commerce

educes transactions costs and increases the transparency of the supply of goods and services (Santarelli and D'Altri 2003).

A precondition for online shops to be a good alternative for consumers is that consumers can trust the information online shops provide about the characteristics of the merchandise, payment conditions, delivery process, privacy and security (Belanger et al. 2002; OECD 2011). Customers of online shops fully depend on the information on the website, while customers of normal (physical) shops can gather information on the quality of products by actually experiencing (e.g., seeing, hearing, feeling) products before making a purchase decision. Moreover, customers of physical shops can deduce the trustworthiness of the shop by visiting the shop and observing and talking to the seller, while the trustworthiness of an online shop is more difficult to assess. So, while consumers always have to deal with imperfect information regarding so-called experience goods and in particular credence goods (Rischkowsky and Doring 2008), this information asymmetry is enlarged when buying such products online. The information asymmetry may result in several inefficiencies, such as consumers buying low-quality products or refraining from buying due to a lack of confidence.

As online sellers have an interest in a positive attitude of customers regarding their merchandise, they have an incentive to provide information that may increase the trustworthiness of their shop. Feindt et al. (2002) find that an informative website that fosters interaction with customers is one of the critical success factors for e-commerce firms. For the market for higher education, for instance, Mause (2010) finds that sellers have incentives to organize a quality information system with the help of private-party intermediaries, making regulatory intervention to protect the buyers on this market unnecessary. This does not mean, however, that online sellers always have an incentive to be as informative as possible. Online sellers in particular have an incentive to design their online shop to maximize consumer expenditures (Belanger et al. 2002). So, regulatory intervention might be needed to realize full transparency in online consumer markets.

Governments can choose from a number of options to reduce the information asymmetry, such as subsidizing private provision of product information, making the disclosure of product information legally

obligatory and protecting consumers against deceptive advertising by sellers (Mause 2010). Regarding online shops registered in the Netherlands, the Dutch government has chosen to implement information-disclosure rules in consumer legislation. The Dutch consumer law stipulates online shops to provide information to customers regarding a number of key characteristics, including the contact details of the firm, the cost structure of purchases, the existence of a time window for reflection (to cancel a purchase) and the processes of ordering, paying and delivery (ACM 2012). Maintaining these rules is delegated to the Consumer Authority. This authority regularly monitors whether firms' behavior is in line with consumer law. If not, the Consumer Authority has the legal power to take punishment measures, including to hand out monetary penalties. The Consumer Authority uses several instruments to influence firm behavior. One of these instruments is submitting guidance, i.e., distributing information on the contents of legal rules, the necessity to obey such rules and the possible legal consequences of non-compliance. In this paper, we assess to what extent such instruments are effective in raising the compliance with information-disclosure rules of e-commerce firms.

Note that improving the effectiveness of the enforcement of the legal rules on information disclosure does not necessarily imply that consumers will make better decisions. Psychological factors, as selective optimism, the endowment effect and bounded rationality, may cause that well-informed consumers make decisions which are not in their self-interest (Faure and Luth 2011). The existence of these factors does, however, not imply that consumers do not benefit from measures which reduce the information asymmetry in consumers markets. Hence, searching for effective regulatory measures to reduce the information asymmetry might increase the welfare of consumers, even if they behave less rationally as is generally assumed in economic models.

3 Method of measurement

3.1 Experimental design

The objective of this study is to determine to what extent three Consumer Authority interventions are effective in enforcing the legal obligations on information disclosure by e-commerce firms. Insofar, these interventions are given on a firm level, and we are able to conduct a randomized field experiment, similar to Collins (2013), Apesteguia et al. (2013) and Iyer et al. (2010), in which the interventions are considered as treatments, while online shops are randomly assigned to a treatment group and a control group. Figure 1 provides an overview of the timing of the three treatments and the various before and after measurements, while Table 2 shows the contents of the treatments. The first firm-specific treatment (Firm Letter Guidance) entails a personalized letter to the legal owner of the online shop in which the Consumer Authority stipulates the need for compliance with the legal framework and informs the owner about possible legal consequences of non-compliance (see ''Appendix 2''). To increase the power of this intervention, the letter is sent by mail in an official envelope of the Consumer Authority. The second firm-specific intervention (Firm Email Guidance) is given about 1.5 year later and consists of sending a personalized e-mail to the firms of the treatment group. This e-mail also included a hyperlink to the website of the Consumer Authority given more detailed information on how the online shop could present the information which is legally required. Note that implementing the legal rules does not require much effort by the online shop, as the rules only refer to mentioning specific information on specific places on the websites.

The third treatment is the Industry Guidance, which is directed to all online shops in this industry. This intervention entailed a range of Consumer Authority activities, including dedicated publications, press releases, presentations, distribution of checklists, journal articles and links on websites of the Chamber

Fig. 1 Overview of the timing of the three treatments and the various measurements

Firm Letter Industry Guidance Guidance Interventions | |

Measurements | 2.5 months

t=0 t=l

i 2.5 months j"

Firm Email Guidance

13 months

2.5 months t = 3 t = 4

Table 1 Contents of three interventions by the Consumer Authority

Treatment (intervention)

Action by the Consumer Authority

Firm Letter Guidance

Firm Email Guidance

Industry Guidance

Letters sent to each firm of the treatment group to inform them about the legal rules, the obligation to meet these rules and the ability of the regulator to give firms a fine if they do not apply with these rules. See ''Appendix 2''

E-mails sent to each firm of the treatment group to inform them again about the legal rules, the obligation to meet these rules and the ability of the regulator to give firms a fine if they do not comply with these rules. The e-mails also included hyperlinks to a web page of the Consumer Authority that showed how online shops could incorporate the legal rules. See ''Appendix 3'' Publication of a document about the legal rules including links to a website of the regulator illustrating the

information online shops should include Press release about the document and the regulator's website. Several major Dutch newspapers published this information

A number of agencies, including the Chamber of Commerce and the Office of Tax Collectors, as well as representative bodies of (web-shop) firms, placed a link on their website to the website of the Consumer Authority where a demo is placed of a well-designed website Consumer Authority representatives gave a number of presentations about the guidance and distributed

checklists during the annual web-shop days in Utrecht Consumer Authority representatives used Twitter and other public media to spread information about the guidance

Launch of free-publicity campaign, which resulted in a number of news items in periodicals of representative organizations of online shops

of Commerce and the Office of Tax Collectors and others to a demo website made by the Consumer Authority demonstrating which information ought to be placed where (see Table 1). As the Industry Guidance is aimed at all online shops, it is an intervention potentially impacting all firms. Because the Industry Guidance is given to the group of all firms, it cannot be analyzed through such a field experiment, and therefore, we apply an econometric panel analysis here.

3.2 Compliance measurements

The data-collection instrument to measure the level of compliance is based on the legal framework that identifies 19 information items that should be available in a website. For each of these items, we developed an operational definition to turn the item into an observable website characteristic. The definitions are developed in collaboration with the Consumer Authority, who also approved the final data-collection instrument. The legal framework not only refers to the availability of an item but also addresses the way as well as the place this element is implemented in a website. For example, the legal framework requires websites to be specific in referring

to an item. With regard to shipping costs, the link or button to this item should be clearly mentioned as, e.g., ''Shipping'' or ''Delivery costs,'' and not be general such as ''Terms and Conditions.'' The items also need to be easily found by consumers. In this example, consumers should not have to click on several subsequent links before they finally find the information on shipping costs. Consequently, the data-collection instrument measures each information item on three dimensions:

• Content: Does the website contain the item?

• Presentation: Is the item presented clearly? (e.g., not as part of a large block of text or as fine print).

• Place: Is the item placed on the right spot in the website?

It is important to acknowledge that the legal framework includes specific rules for each of these dimensions, implying that each of them can be monitored independently. The only exception is that Presentation and Place cannot be monitored if a website does not include the item. When, in such cases, Presentation and Place cannot be observed, the data are coded as missing.

Each of these three characteristics is measured in a binary form: The online shop either complies with the

legal rule or does not. Compliance is coded as 1 and non-compliance as 0. By aggregating over the three characteristics, we compute a compliance index, which reflects the percentage to which an online shop complies with the legal rules for an item (between 0 and 100 %). By using these observations, an index (Item Index) is computed that reflects the percentage of online shops complying with the three characteristics of Content, Presentation and Place:

El=iE S= iE M

Overall Index

■d= 1 Misd

Item Indexi

ES=1 Ed=1 Misd Es=1 E<i=1 Nisd

In this formula, i stands for items (1___19), d for the

dimensions Content, Presentation and Place, s for online shops (1 _S), while N refers to the existence of an observation (its sum represents the number of non-missing observations) and M to the measurement outcome (0 or 1). Hence, by this formula, we have a score for each item for the total group of S online shops. The lowest possible index value is zero, indicating that no online shop complies on any of three dimensions. If all online shops would fully comply on all three dimensions, the value is 1 which is the maximum value of this index.

Similar to the computation of the Item Index, we also compute a Category Index. For that purpose, the 19 items are grouped into the following six categories:

1. Firm characteristics: e.g., firm name, registration number and address (5 items).

2. Additional costs: e.g., taxes and delivery (3 items).

3. Reflection period: e.g., whether it is offered and of sufficient duration (4 items).

4. Order process: e.g., overview of stages and products ordered (4 items).

5. Payment process: timing and deadline (2 items).

6. Delivery process: delivery method (1 item).

The formula for calculating a Category Index (c = 1_6) is:

Category Indexc

Ei=1 E S=1E 3=1 Mcisd Ei=1 E S=1 E<i=1 Ncisd

where Ic refers to the number of items within category c.

Finally, we calculate an Overall Index:

Ei'=1 E S=1 E 3=1 Nisd

All tables in the next sections report percentages in terms of either the Item Index, the Category Index or the Overall Index. We use the scores on these indexes to assess the effectiveness of the guidance given by the Consumer Authority.

3.3 Panel analysis of Industry Guidance

In order to determine the impact of the Industry Guidance, we cannot analyze the differences between the treatment group and the control group as this guidance is provided to all firms (see Table 1). Therefore, we analyze changes in behavior over time for both groups controlling for other factors which may influence firm behavior, including the firm-specific guidance given to firms in the treatment group. Panel analysis enables us to determine the change in compliance between the measurement just before the Industry Guidance was released and the measurement about 2.5 months later. The panel analysis is conducted at the level of the Category Indexes. Note that controlling for other factors was not needed in our previous analyses as the firms in the treatment group and the control group have similar characteristics while operating in the same environment.

We estimate the effect of the Industry Guidance by using a dummy: This dummy is set equal to 1 for all observations after the Industry Guidance was given, and 0 for all observations at previous time periods. We include a dummy to control the influence of the firm-specific guidance: This dummy equals 1 for all observations of the firms in the treatment group, and 0 for the control group. In addition, we control the age and the legal status of the online shops by including these variables directly as well as by including the interaction terms between these variables and the dummies for the firm-specific and the guidance on industry level. The latter is done because older firms and firms with a formal legal position may show better compliance with legal obligations. We include the legal status by a dummy which is 0 for firms which are a sole-trader business and 1 for all other firms. In order to control general time effects, we include a trend variable.

3.4 Sampling

We conduct our analysis on the industry that exploits online shops specialized in selling cloths and luxury articles. Regarding consumer online purchases, this industry is one of the most important online industries. A recent report identifies this industry as being the largest in terms of online sales after telecom subscriptions3. Total 2012 sales are estimated at 730 million euro, up 16 % from 2011, and up 500 % compared to 2004. The sampling frame consists of all 916 online shops registered in this industry at the Dutch Chamber of Commerce on September 11, 2012. This group represents the population of this type of firms in the Netherlands.

The Chamber of Commerce data are used to sample both the treatment group and the control group. Both samples are randomly selected and compared with each other and the sampling frame in terms of age, legal status and geographical region. Age is considered to be important as online shops are a relatively new phenomenon, growing rapidly in numbers, with a 50 % increase between 2010 and 2012 (Thuiswinkel. org 2013), but having a low survival rate, reportedly only 41 % after 4 years.

Determining the right sample size involves making the trade-off between statistical and practical considerations. Practical considerations refer to the efforts and costs of data collection, and the fact that this study involves two groups (two samples) and five measurements for each online shop. Therefore, we randomly selected 150 firms for both the treatment group and the control group, which results in a confidence level of 92 %.6 Next, we checked whether the firms selected

3 Webwinkels/Online-bestedingen-per-product.html?subonder-werp_id=67



6 The confidence level (z) follows from the size of the sampling frame (N = 916 firms), the standard deviation (r) around 6 age classes in this group (0-1.5 years, 1.5-3 years, 3-4.5 year, 4.5-6 years, 6-7.5 years and 7.5 and more years), the means of both the sampling frame (i) and the sample (X) and the size of the sample (Malhotra 2007):

from the Chamber of Commerce actually represent active online shops, by visiting each of the 300 website addresses. A considerable proportion of the online shops listed in the database turned out to be unavailable, 31.7 % of the 300 online shops. Therefore, we randomly added additional firms to both samples and checked these websites, until we reached 150 observable online shops for both the treatment group and the control group. In ''Appendix 1,'' we describe the background statistics of the online shops in the population, the sample, the treatment group and the control group.

4 Results

4.1 Initial compliance

The first analysis focuses on the initial compliance of the online shops in both the treatment group and control group (at t = 0). Table 2 shows that about 80-90 % of the online shops score well on the requirements regarding the categories Firm characteristics, Additional costs and Order process, while almost all firms fulfill the requirements regarding the Delivery process. Compliance is, however, relatively low on information regarding the Reflection period (30 %) and the Payment process (30). Overall, non-compliance is not extensive but considerable which is indicated by the scores on the Overall Index of just above 75 %. Overall, these findings suggest ample room for improvement, which supports the idea of interventions by the Consumer Authority.

A t test is used to compare the treatment group with the control group. Out of the 25 tests (19 items and 6 categories), only two differences are statistically significant. In both cases, it concerns an Item Index for which the control group mean is higher than the mean for the treatment group (E-mail address and Direct contact possibility). Overall, we conclude that the initial level of compliance of both groups is similar.

4.2 Effects of Firm Letter Guidance

Immediately after the measurement of the initial compliance, we send all firms in the treatment an official personalized letter on behalf of the Consumer Authority in order to inform them about the legal rules,

Table 2 Initial compliance, by both the treatment group and control group (index in %)

* Refers to a significant difference between the treatment and control group (p < 0.05)

the obligation to meet these rules and the ability of the regulator to give firms a fine if they do not comply with these rules (see ''Appendix 2''). After 2.5 months, we measured again the compliance of all firms in both groups. The period of 2.5 months was chosen to offer online shops sufficient time to respond to the treatments by changing their website.

Table 3 shows the performance of the firms in the two groups. We test whether the changes in both groups are different by applying a one-tailed t test. Assuming the Firm Letter Guidance has an effect, we may expect larger positive changes in the treatment group than in the control group. The treatment group does slightly better in t = 1 compared to t = 0 in terms of Category Indexes, though the largest increase is still only 3 %. Compared to the control group, the treatment group shows a significantly larger increase in terms of one Category Index, namely Additional costs. Regarding the 19 Item Indexes, for only one of

Category/Item Index Treatment group Control group

Firm characteristics 85.3 88.9

Firm name 93.1 91.3

Registration number 65.1 71.3

Address in real world 64.4 69.1

E-mail address 68.0 79.6*

Direct contact possibility 88.0 94.7*

Additional costs 80.1 79.9

Taxes 58.9 59.3

Delivery costs 82.9 85.3

Other unavoidable costs 99.8 99.8

Reflection period 31.6 33.4

Period for reflection 32.2 35.8

Duration of at least 7 days 20.2 19.8

Why reflection not relevant 34.7 36.7

Reference to legal exemption 34.7 36.7

Order process 78.7 79.2

Overview process stages 53.1 55.3

Option to reconsider 0.4 0.0

Overview ordered products 100 98.0

Option to change order 82.0 84.0

Payment process 32.2 27.5

Before/after delivery 42.0 36.4

Deadline after delivery 2.7 0.7

Delivery process 99.3 99.3

Delivery method 99.3 99.3

Overall Index 76.1 77.2

them (Overview of the process stages) we find a significantly larger increase for the treatment group. Overall, however, we find that the effect of the firm-specific guidance by sending firms a personalized letter by post mail is not significant in terms of most of the in total 25 indexes. This is also indicated by the Overall Index which shows no significant difference between the treatment and the control group.

4.3 Effects of Firm Email Guidance

The second firm-specific intervention consists of sending a personalized e-mail to all firms of the treatment group about 1.5 year after they had received the post mail letter. This e-mail included the same information as the letter plus a hyperlink to the website of the Consumer Authority where a demo of a well-designed online shop was presented. As the e-mail was sent to the same group of firms which also received the letter, the

Table 3 Effect of the Firm Letter Guidance (numbers are differences in index values)

Category/Item Index

(Difference t = 1 and t = 0)

Treatment group

Control group

na not available * Significant difference between the treatment group and the control group (5 %, t test, one-tailed)

Firm characteristics 1.9 -0.2

Firm name 1.0 3.5

Registration number 5.2 -4.3*

Address in real world 4.0 0.7

E-mail address 3.0 -2.1*

Direct contact possibility 2.2 1.4

Additional costs 1.7 0.2

Taxes 0.7 0.7

Delivery costs 3.2 -0.2

Other unavoidable costs 0.0 0.2

Reflection period -0.5 0.8

Period for reflection 0.0 -0.2

Duration of at least 7 days 0.7 1.4

Why reflection not relevant -1.5 0.0

Reference to legal exemption -1.5 0.0

Order process 3.6 1.1*

Overview process stages 10.0 -1.4*

Option to reconsider 11.4 8.5

Overview ordered products 0.0 2.1

Option to change order 1.5 -2.1

Payment process -0.2 1.5

Before/after delivery 0.0 1.7

Deadline after delivery 0.7 0.7

Delivery process 0.0 0.7

Delivery method 0.0 0.7

Overall index 2.3 0.6

performance of this group relative to the performance of the control group has to be attributed to both interventions. Hence, this effect must be seen as the sum of the long-term effect (after about 1.5 year) of sending the personalized letter by post mail and the short-term effect (after 2-3 months) of sending a personalized e-mail. Using the same treatment group for both interventions can influence the results in multiple ways. It could have had a positive effect on compliance, if the second firm-specific intervention has served as a reminder to adjust the website in order to comply with legal obligations. The effect could also be negative, if, after not making the required changes after the first intervention, the firms would have experienced no legal punishment, thereby learning that they can safely ignore such regulatory interventions.

It appears that the absolute changes for the treatment group are most often small (see Table 4). In terms of

Category Indexes, all changes for this group are in the range of -0.0 and 2.6 % points. None of the Category Indexes shows a significant difference between both groups. We find only one significant difference between both groups when considering the Item Indexes, notably for information on taxes. Overall, again, we do not observe a statistically significant difference in performance between both groups, meaning that there is no proof that the joint effect of the two different forms of firm-specific guidance differs from zero.

4.4 Effects of Industry Guidance

As firms may respond differently to different kinds of regulatory intervention, we also included guidance on industry level in our research. This guidance is given to all online shops by a number of dedicated publications and presentations by the Consumer Authority.

Treatment group Control group

-0.0 0.7

3.3 2.8

-1.4 1.2

-0.7 2.8

0.0 0.0

1.1 0.0

0.4 0.4

1.8 -1.6**

1.1 1.9

0.0 0.0

2.0 2.1

2.2 1.6

0.4 1.9

3.3 3.7

3.3 3.7

1.1 1.0

3.3 5.6

-1.1 -2.8

0.0 0.0

2.5 2.8

2.6 4.0

3.3 4.7

1.4 1.9

0.0 0.0

0.0 0.0

1.2 1.6

Firm characteristics Firm name Registration number Address in real world E-mail address Direct contact possibility Additional costs Taxes

Delivery costs Other unavoidable costs Reflection period Period for reflection Duration of at least 7 days Why reflection not relevant Reference to legal exemption Order process Overview process stages Option to reconsider Overview ordered products Option to change order Payment process Before/after delivery na not available Deadline after delivery

* Significant difference Delivery process

between the treatment Delivery method

group and the control group Overall Index (5 %, t test, one-tailed)

The effects of the Industry Guidance are measured by applying panel analysis. Table 5 shows that the Industry-Guidance dummy has no significant effect on the compliance with legal rules by the online shops. For all indexes, the coefficients of the Industry-Guidance dummy are statistically not different from zero. The panel analysis also shows that the firm-specific guidance has neither a significant effect on the compliance, thereby confirming the previous analyses.

In one case (Payment process), the age of the online shops has a positive influence on compliance. This implies that older online shops comply to a larger extent with legal obligations with respect to information on this category. In another case (Order process), the age of the online shops fosters the compliance after the Industry Guidance. The results also show that the legal status has some effect on compliance. Firms with a more formal legal status (not being a ''sole-trader business'') comply better with the legal rules. Overall,

firms with a formal legal status have on average a 4 % point higher performance than sole-trader firms. In addition, we find that the performance of the online shops improves slightly over time, given the consistently positive coefficient of the trend, albeit this coefficient is only significant in one case (Order process). This is also shown by the constants of the panel regressions which are almost equal to the average values of the dependent variables at t = 0 (see Table 2), which again highlights that the explanatory power of the explanatory variables is poor.7

5 Conclusions

Consumer markets are known to be vulnerable for problems of information asymmetry, resulting in

7 The R squared of the panel model is about 2 % in all cases.

Table 4 Effect of the Firm Email Guidance (numbers are differences in index values)

Table 5 Results of Industry Guidance (number of groups = 300; number of periods = max 4; number of observations = 1124)

Value of Category Index Information on

Firm Costs Time for reflection Order process Payment process Delivery process Overall Index

Constant 0.86*** 0.75*** 0.25*** 0 7g*** 0.31*** 0 99*** 0.75***

(0.02) (0.02) (0.05) (0.02) (0.06) (0.008) (0.01)

Dummy Industry Guidance -0.008 -0.02 0.04 -0.03 -0.03 -0.005 -0.01

(0.01) (0.01) (0.03) (0.01) (0.03) (0.004) (0.01)

Dummy Firm Letter Guidance 0.01 -0.008 -0.001 0.01 0.11 -0.009 -0.0001

(0.03) (0.03) (0.08) (0.02) (0.08) (0.01) (0.02)

Age firm 0.000009 -0.000006 0.00001 -0.000002 0.00004* -0.000002 0.000003

(0.000007) (0.000007) (0.00002) (0.000006) (0.00002) (0.000003) (0.000005)

Age firm x Firm Letter Guidance -0.00002 -0.00002 0.00001 -0.000005 -0.00003 0.000002 0.000002

(0.00002) (0.00002) (0.00004) (0.00001) (0.00004) (0.000006) (0.00001)

Age firm x Industry Guidance 0.000001 0.000003 -0.00001 0.00002*** -0.000005 0.000002 0.000002

(0.000004) (0.000004) (0.00001) (0.000004) (0.00001) (0.000002) (0.000002)

Legal status 0.06* 0.01 0.16** 0.03 -0.03 0.002 0.04***

(0.03) (0.03) (0.08) (0.02) (0.09) (0.01) (0.02)

Legal status x Firm Letter Guidance -0.06 -0.003 -0.05 0.03 -0.14 0.006 -0.01

(0.04) (0.04) (0.12) (0.04) (0.14) (0.02) (0.03)

Legal status x Industry Guidance -0.003 0.02 -0.07 -0.005 -0.07 -0.001 -0.01

(0.01) (0.01) (0.03) (0.01) (0.04) (0.006) (0.01)

Trend 0.004 0.005 0.002 0.01*** 0.004 0.002 0.005

(0.003) (0.003) (0.008) (0.003) (0.009) (0.001) (0.002)

The total number of observations is less than 1200 (300 firms x 4 measurements), as some online shops which were included at t = 0 did not exist anymore at one or more of the following measurements. If we restrict the panel analysis to those online shops that are present in all four measurements, we find similar results Between brackets the standard error; *, ** and *** refer to significance levels of 10, 5 and 1 %, respectively

inefficient market outcomes. This holds in particular for online consumer markets due to the limited opportunities to experience merchandise before making a purchase decision. Because of this market failure, governments may implement a number of regulatory measures, including legal obligations regarding the pre-purchase information firms need to disclose. The effectiveness of such obligations depends on the strength of enforcement. In this paper, we assess the effectiveness of enforcing the legal obligations on online shops, by three regulatory interventions, each informing the SMEs in a different way about the content and importance of legal obligations.

We find that the initial level of non-compliance of the online shops is considerable. Giving these firms guidance through the personalized letters has a small effect on compliance in the short term. In the longer term, however, this effect disappears. Also the Industry Guidance has hardly any effect on firm compliance behavior. Our study design with an experimental as well as a control group plus the use of panel analysis enabled us to control many factors that could possibly bias the results. We therefore conclude that the lack of effects results from the ineffectiveness of the regulatory interventions. Although information-related interventions are commonly used, our research suggests that regulatory interventions in the form of providing only guidance to firms can be ineffective in influencing their behavior. Firms do not seem to change their behavior in response to just receiving more information on legal obligations. Although both letters, which were personalized and sent by the Consumer Authority to each firm in the treatment group, mention the legal power of the authority to fine non-complying firms, this threat is apparently not strong enough to have firms adapt their behavior.

This result differs from Apesteguia et al. (2013) who concluded that sending e-mail messages with information regarding rules helps to promote compliance. One possible reason for the marked differences in intervention effectiveness is related to differences in the economic agents (SMEs vs. individuals), since firms may make a more rational analysis of the expected costs of violating rules (Telle 2009, 2013). If firms are already well aware of the legal requirements, any additional information on these rules can be expected not to have any additional effect. Insofar, firms were not well informed, and additional information may only raise compliance if the (rational) firms expect that the marginal benefits of

compliance exceed the marginal costs. Note also that the administrative costs of compliance are viewed to be small since only a limited number of information needs to be given on specific places within the websites. Since the guidance was restricted to giving information on the rules while the ability of the Consumer Authority to fine firms for not complying was mentioned but not stressed very much, the expected marginal benefits of compliance may be negligible as well. Hence, not responding to the information provided by the Consumer Authority may be a rational firm strategy.

More research is needed to explain the absence of almost any effect of the information interventions aimed at firm compliance. Further research could focus on the reasons why e-commerce firms, which are usually small and young firms, do hardly respond to information on the legal obligations. As two interventions included a personalized letter to the firm, not being aware of the legal obligations is unlikely the reason for non-compliance behavior in our study. A more in-depth understanding of non-compliance behavior will help regulatory authorities in designing effective interventions. Further research could also focus on the timing of the measurements; we measured compliance about 2.5 months after an intervention to allow the online shops enough time to make the required changes, and further research could explore how much time firms actually need for implementing such changes.

Based on our findings, we formulate the following three implications for regulatory authorities. First, although regulatory authorities need to inform SMEs about legal obligations, they also need to realize that just informing them about the legal rules may be not sufficient to change their behavior. Adding firm-specific information about their actual individual compliance with these rules is a potential improvement of such a strategy. In addition to communicating legal obligations and the regulator's ability to fine, serious regulatory threats are likely needed in order to have a significant influence on firm behavior (Iyer et al. 2010).

Second, regulatory authorities may consider more creative interventions that could be effective if they have important consequences for the economic agents. For example, the regulatory authority in our study may consider involving consumer advocate organizations and even individual consumers in interventions. Consumer advocate organizations could report scores (rankings) of firms complying with a varying extent to legal obligations (e.g., by using our measurement

model), while consumers could adjust their purchase behavior in response to such information. The resulting bad publicity and missing sales may be effective in changing firm behavior (see De Witte and Saal 2010).

Third, research into the effectiveness of government interventions may benefit from adopting our randomized field experimental design. Following and extending previous effectiveness studies based on field experiments (e.g., Apesteguia et al. 2013; Collins 2013; Iyer et al. 2010), we have shown that such a design is a powerful, accurate and realistic way to study how effective interventions are in both the short and long terms. The distinction between a treatment group and control group allows researchers to control changes in the environment that are beyond control of the researcher but that could affect intervention effectiveness.

Although regulatory interventions are found in many situations and in many forms, we still lack knowledge with regard to their effectiveness and efficiency (Niels and Van Dijk 2008). This paper provides clear evidence that at least not all interventions aimed at SMEs are effective. The information-related interventions were hardly effective in changing the behavior of e-commerce firms. In combination with the importance of regulatory interventions for national economic policies and the current lack of understanding of their effectiveness, more field research is needed to explore this issue.

Acknowledgments The authors thank Klaas van der Veen and Sigourney Zomer for their research support. The authors are also grateful for the support given by the Authority for Consumers and Markets (ACM). The authors are fully responsible for any remaining shortcomings. This paper does not constitute any obligation on the ACM.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Appendix 1: Background characteristics of online shops

Table 6 shows that the total sample (300 online shops) is comparable to the population in terms of age, legal

status and region (Chi-square tests, p > 0.05). Similar results are found when comparing the treatment group and the control group with the population data.

Appendix 2: Text of letter sent to treatment group

Dear online shop "...,"

According to our information you own a online shop. This is the reason you receive this letter.

The Consumer Authority is responsible for the regulatory supervision on compliance with the consumer law. This year the Consumer Authority directs her attention to online shops since the number of complaints about these shops by consumers has increased.

Online shops are subject to legal rules which are meant to protect consumers. By this letter, the Consumer Authority informs you about these rules since they enable you to guarantee that consumers can trustfully buy on the internet. This is good for consumers and also for you, as consumers will become more active on the internet if they know they can trust online shops.

In the appendix to his letter, you find a concise description of the key legal rules, including some additional information. The appendix also shows you how to design your online shop which is in line with the legal rules.

Does your online shop not comply with the legal rules? Please, make the necessary adaptations as soon as possible. Note that the Consumer Authority has the legal power to fine online shops which do not comply with the legal rules.

More information on the Consumer Authority can be found on

Yours sincerely,

The Consumer Authority.

Appendix 3: Text of e-mail sent to treatment group

Dear online shop "...."

Do you know that your online shop needs to comply with a number of legal obligations?

These legal rules are not without purpose. They are meant to provide a trustworthy online environment where consumers are able to buy products without any difficulty.

Table 6 Characteristics of population, sample, treatment group and control group (in %)

Chi-square tests show that the sample is similar to the population and the treatment group to the control group on all three characteristics (p > 0.05)

Characteristic Population Sample Treatment group Control group

Age (year)

<1.5 19.5 19.7 21.3 18.0

1.5-3 24.3 27.0 26.7 27.3

3-4.5 18.7 19.0 21.3 16.7

4.5-6 17.7 17.7 18.7 16.7

6-7.5 8.1 7.7 4.7 10.7

>7.5 11.7 9.0 7.3 10.7

Total 100 100 100 100

Legal status

Sole-trader business 84.0 82.3 85.3 79.3

Partnership 12.0 14.0 12.0 16.0

Other 4.0 3.7 2.7 4.7

Total 100 100 100 100

Geographical region

Zuid-Holland 17.1 17.7 18.0 17.3

Noord-Holland 15.5 18.7 16.7 20.7

Noord-Brabant 12.9 12.7 14.7 10.7

Gelderland 12.2 12.7 11.3 14.0

Overijssel 8.7 6.0 5.3 6.7

Utrecht 8.1 9.7 10.7 8.7

Drenthe 5.0 4.3 4.0 4.7

Flevoland 4.9 3.0 2.7 3.3

Groningen 4.8 4.0 4.7 3.3

Friesland 4.1 5.0 4.7 5.3

Limburg 4.1 5.0 5.3 4.7

Zeeland 2.4 1.3 2.0 0.7

Total 100 100 100 100

The Consumer Authority monitors compliance with these rules. On the website8 of the Consumer Authority, the rules are outlined. Also suggestions are given on how to improve your online shop.

Does your website not comply with these legal rules? Please, improve your website as soon as possible. It will only take little effort to change your online shop, while it is important for both consumers and your compliance to regulation.

More information on the Consumer Authority can be found on

Yours sincerely,

The Consumer Authority.

8 The address of the website is: onderwerpen/verkoopmethode/webwinkels/regels-voor-web-winkels/.


Apesteguia, J., Funk, P., & Iriberri, N. (2013). Promoting rule compliance in daily-life: evidence from a randomized field experiment in the public libraries of Barcelona. European Economic Review, 64, 266-284.

Authority for Consumers & Markets (ACM) (2012). Con-sumentenautoriteit agenda 2012-2013, The Hague.

Belanger, F., Hiller, J. S., & Smith, W. J. (2002). Trustworthiness in electronic commerce: The role of privacy, security and site attributes. Journal of Strategic Information Systems, 11, 245-270.

Bergman, M. A. (2008). Quis custodiet ipsos custodies? Or measuring and evaluating the effectiveness of competition enforcement. De Economist, 156, 387-409.

Collins, J. M. (2013). The impacts of mandatory financial education: Evidence from a randomized field study. Journal of Economic Behavior & Organization, 95, 146-158.

de Witte, K., & Saal, D. S. (2010). Is a little sunshine all we need? On the impact of sunshine regulation on profits, productivity and prices in the Dutch drinking water sector. Journal of Regulatory Economics, 37, 219-242.

Don, H., Kemp, R., & van Sinderen, J. (2008). Measuring the economic effects of competition law enforcement. De Economist, 156, 341-348.

Faure, M. G., & Luth, H. A. (2011). Behavioral economics in unfair contract terms: Cautions and considerations. Journal of Consumer Policy, 34, 337-358.

Feindt, S., Jeffcoate, J., & Chappell, C. (2002). Identifying success factors for rapid growth in SME E-commerce. Small Business Economics, 19, 51-62.

Grandon, E. E., & Pearson, J. M. (2004). Electronic commerce adoption: An empirical study of small and medium US businesses. Information & Management, 42, 197-216.

Iyer, G. S., Reckers, P. M. J., & Sanders, D. L. (2010). A field experiment to explore the effects of detection and penalties communications and framing among Washington State retail firms. Advances in Accounting, 26(2), 236-245.

Levitt, S. D., & List, J. A. (2009). Field experiments in economics: The past, present and the future. European Economic Review, 53, 1-18.

Malhotra, N. K. (2007). Marketing research an applied orientation (5th ed.). New Delhi: Prentice-Hall Of India Pvt.

Mause, K. (2010). Considering market-based instruments for consumer protection in higher education. Journal of Consumer Policy, 33, 29-53.

Nicoletti, G., & Pryor, F. L. (2006). Subjective and objective measures of governmental regulations in OECD countries. Journal of Economic Behavior & Organization, 59, 433-449.

Niels, G., & van Dijk, R. (2008). Competition policy: What are the costs and benefits of measuring its costs and benefits. De Economist, 156, 349-364.

Nosenzo, D., Offerman, T., Sefton, M., & van der Veen, A. (2013). Encouraging compliance: Bonuses versus fines in

inspection games. Journal of Law Economics and Organization, 30(3), 623-648.

OECD. (2011). Consumer protection in e-commerce payments, in: The role of internet intermediaries in advancing public policy objectives. Paris: OECD Publishing.

OECD. (2013). Technology and industry scoreboard 2013. Paris: OECD Publishing.

Rischkowsky, F., & Döring, T. (2008). Consumer policy in a market economy: Considerations from the perspective of the economics of information, the new institutional economics as well as behavioral economics. Journal of Consumer Policy, 31, 285-313.

Santarelli, E., & D'Altri, S. (2003). The diffusion of E-commerce among SMEs: Theoretical implications and empirical evidence. Small Business Economics, 21, 273-283.

Stafford, S. L. (2003). Assessing the effectiveness of State regulation and enforcement of hazardous waste. Journal of Regulatory Economics, 23(1), 27-41.

Telle, K. (2009). The threat of regulatory environmental inspection: Impact on plant performance. Journal of Regulatory Economics, 35, 154-178.

Telle, K. (2013). Monitoring and enforcement of environmental regulations; lessons from a natural field experiment in Norway. Journal of Public Economics, 99, 24-34.

Terzi, N. (2011). The impact of e-commerce on international trade and employment. Procedia Social and Behavioral Sciences, 24, 745-753.

Wölfl, A., Wanner, I., Kozluk, T. & Nicoletti, G. (2009). Ten years of product market reform in OECD countries; Insights from a revised PMR indicator. OECD Economics Department Working Papers 695.