Scholarly article on topic 'Trust Necessitated through Metrics: Estimating the Trustworthiness of Websites'

Trust Necessitated through Metrics: Estimating the Trustworthiness of Websites Academic research paper on "Computer and information sciences"

CC BY-NC-ND
0
0
Share paper
Academic journal
Procedia Computer Science
OECD Field of science
Keywords
{"Web Trust" / Credibility / "Content Trust" / "Quantization of Trust" / "User Behavior" / "User Satisfaction."}

Abstract of research paper on Computer and information sciences, author of scientific article — Himani Singal, Shruti Kohli

Abstract The issue of trust is of exhilarating concern in case of information disseminated through websites. Digitization technologies such as Internet has lowered the cost and increased the access to information of all kinds. This increased exposure to information from humongous number of websites poses a need for some mechanism or tool that can address this issue and provide rankings to the websites on basis of their credibility or the trust value they possess. Previous research has shown a lot of efforts in this area. Both exogenous and endogenous signals have been evaluated to derive the credibility of websites. Trust being more of a sociological virtue of psychometric nature; cannot be quantified easily. Aiming this fact we have tried to measure the trust by evaluating the actual behavioral metrics of the users (of the websites) which gets collected by analytical tools automatically with every hit on a website. This gives a pertinent and bias-free quantification of trust value of a website as tools collect the data being anonymous from the user. In this paper, we have introduced our novel web-based tool which estimates trust (of websites) imposed by web metrics collected through similarweb.com. The tool will re-rank the web-links (URLs) extracted from Google search engine for any keyword according to the trust maintained by the actual users of the websites. We will also evaluate the performance of our tool in terms of validated rankings and benchmarking it with other recently acceptable tools.

Academic research paper on topic "Trust Necessitated through Metrics: Estimating the Trustworthiness of Websites"

Available online at www.sciencedirect.com

ScienceDirect

Procedía

CrossMark

ELSEVIER

Procedía Computer Science 85 (2016) 133 - 140

Computer Science

International Conference on Computational Modeling and Security (CMS 2016)

Trust Necessitated through Metrics: Estimating the Trustworthiness of Websites

Himani Singal*, Shruti Kohli

BITMesra, Ranchi-835215, India

Abstract

The issue of trust is of exhilarating concern in case of information disseminated through websites. Digitization technologies such as Internet has lowered the cost and increased the access to information of all kinds. This increased exposure to information from humongous number of websites poses a need for some mechanism or tool that can address this issue and provide rankings to the websites on basis of their credibility or the trust value they possess. Previous research has shown a lot of efforts in this area. Both exogenous and endogenous signals have been evaluated to derive the credibility of websites. Trust being more of a sociological virtue of psychometric nature; cannot be quantified easily. Aiming this fact we have tried to measure the trust by evaluating the actual behavioral metrics of the users (of the websites) which gets collected by analytical tools automatically with every hit on a website. This gives a pertinent and bias-free quantification of trust value of a website as tools collect the data being anonymous from the user. In this paper, we have introduced our novel web-based tool which estimates trust (of websites) imposed by web metrics collected through similarweb.com. The tool will re-rank the web-links (URLs) extracted from Google search engine for any keyword according to the trust maintained by the actual users of the websites. We will also evaluate the performance of our tool in terms of validated rankings and benchmarking it with other recently acceptable tools.

© 2016 The Authors.Publishedby ElsevierB.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-reviewunder responsibility of the Organizing Committee of CMS 2016

Keywords: Web Trust; Credibility; Content Trust; Quantization of Trust; User Behavior; User Satisfaction.

1. Introduction

When a search query is entered in a search engine, the user is returned with a large number of web-links (URLs) to websites claiming to contain the related information. With such huge pool of web-links available, user does not

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of the Organizing Committee of CMS 2016

doi:10.1016/j.procs.2016.05.199

know which of them to select on the basis of trusted or credible information, as the origination, motivation and latent purpose of their information dissemination is unknown [1].

It is advocated that since the information is present on the Internet in an analogous arrangement of Websites, an even layout of information is laid as far as accessibility is concerned, thus giving almost equal credibility to all the authors in the Internet users' observances [2-7]. Information on the web is generally not filtered using any professional mechanisms and it is largely deficit of "traditional authority indicators" viz. identity and reputation [2]. There are no universal or globally accepted standards for publishing online and the information may be effortlessly copied, altered or produced namelessly under false facades. Thus a user-based judgment is essential before consuming any kind of information from websites which can be both objective (information quality, accuracy, etc.) and subjective (expertise of author, attractiveness of website, etc.) in nature [8]. It is claimed that three factors influence the online information extensively viz. credibility of website, simplicity in usage and interest of the users being respected [9]. Trust in websites is vital if the user needs the information for supplementing decision-making as in symptom analysis using information from health related websites.

Our endeavor is to model the actual judgment of website users to find the trust possessed by the websites. As the behavior of each and every user of a website is recorded by the Analytics tools; their cumulative analysis will give most appropriate trust evaluation for the website. This forms the underlying basis of our model which is nomenclature as TNM (Trust Necessitated through Metrics). The model is validated and a web-based tool is built for trust evaluation as accurate as possible.

The rest of the paper is divided into different subsections. The first subsection gives a review of recent and relevant research in the area and background of the work. Then design and implementation of TNM tool is discussed followed by its empirical evaluation which will establish the effectiveness of the tool. Future perspectives for the research are also listed.

2. Background and Related Work

2.1. User Perception

The trust of users in a website depends upon three basic notions as suggested by 3S model in [10]. 3S model specifies semantic features (neutrality, accuracy, etc.), surface features (website quality, design, font size, etc.) and source of information (previous experience with the website) as the three antecedents of credibility assessment of a website. It has been found many a time in studies that these semantic and surface features affects the trustworthiness or credibility of a source only for acquainted or frequent users [10-12]. It may be invalid for all the websites as suggested in [13] but the work was restricted only to Wikipedia articles. An extensive literary assessment from multiple research articles [14-19] led to the determination of precursors of trust in websites enlisted in Table 1.

Table 1: Literary assessment of precursors of trust [14-19]

S. No. Criteria Char acteristics/Implication

1. Purpose of the Website e-commerce, support, lead generation, informational website, etc.

2. Last updated date Website is presumed more credible if it is recently updated.

3. Primary or Secondary source Secondary source is supposed for manipulation of facts of information from

of information primary resource.

4. Contact information Address or e-mail id of author entrusts the user to believe in the information.

accessibility

5. Link integrity of the website No broken links should be there. If the number of external links is larger, then

the credibility of the referred information needs to be assessed.

6. Affiliation Trust is judged on the basis of logo of concerned organization or sponsors by

analyzing the header and footer.

7. Completeness, accuracy and This strengthens the belief of users about credible or trustworthy information.

unambiguity

8. Expertise of author If the provided content is expertise area of the author, then it is considered

trustworthy without any further verification.

9. Website Design Aesthetics of a website has direct link with the acceptable trustworthiness of

the website.

10. Interactivity and Usability As the navigation becomes easy, user's perception of credible information increases.

11. Tone and Quality of Website Overblown arguments, humorous, iconic or exaggerated.

12. Ranking of website in Search This is related directly with the popularity of website among its users.

Engines

13. Domain experts' view Trustworthiness and expertise go hand in hand. Recommendations made by domain experts are followed by novice users. Any special software requirement may limit the access to web information;

14. Software requirements thus lowering the interest of the user to explore the information thus lowering its credibility.

2.2. Process Automation

There have been numerous attempts to automate the process of trustworthiness or credibility assessment of websites. Some are generic as they are applicable to any of the websites while some are for specific set of websites [20-24]. These processes or tools calculate credibility or trust scores by taking weighted mean of different features, by analyzing external or internal links to the websites or searching for alternate information on the internet [25]. Popularity on social networking websites such as Facebook and Twitter is also used for the same [24]. In [21], search results are repositioned according to their relative credibility score based on some of the criteria listed above. These tools have used either human judges or available dataset as their benchmark. On the basis of correlation, validation of these tools has been accepted.

2.3. Gap Analysis

None of the present tools or models of proposed automated process have used the actual behavior of the users of the websites as an aggregator to find the trust scores or ranking of the websites. By cumulative collection of every user on any website, trust scores can be improved substantially as users are the best judges. Journal of Neuroscience claims that "humans can judge the trustworthiness in three hundredths of a second, less than an eye blink" [26]. Though it is said in context of human-human interaction, but it is applicable in human-computer interaction too where three hundredth of a second is replaceable by 30 seconds [27]. Users, over time and with experience, forms their own criteria of trusting information, and seldom does instinct lead one wrong [26-27]. Authors have tried to evaluate and quantize this behavior of website users to build a tool which can categorize websites according to the information credibility or trust value they possess.

3. Design and Implementation

An ideal tool for measuring trustworthiness of a website should consider all the criteria enlisted in previous section, but practically it is not possible to incorporate all of them as there are no direct measures to quantify them. To conquer this problem, a model based on human behavior was proposed by authors in [28] which aggregated the behavior of actual users of the website. It was validated through a questionnaire among internet users [29]. On the basis of this detailed analysis, suitable metrics (collected by Web analytics tools) were identified as antecedents of trust in informational websites [30]. These metrics are:

Average Time on Website: 'Average time spent by a user on website'.

Pages/Visit: 'Average number of web pages visited by a user in a single session'.

Average Daily Visits: 'Amount of traffic or amount of users who have used the website'.

Bounce Rate: 'Number of users who have left the website within ten seconds of their arrival'.

As the trust quantification is being targeted from metrics, the tool is named as TNM (Trust Necessitated through Metrics). To develop the tool; API (Application Programming Interface) programming is used extensively for interfacing with real-time databases. The process diagram of TNM tool is represented in Figure 1.

TNM Tool

Figure 1: Process Diagram of TNM

3.1. Data Collection

Google's Custom Search API [31] is used for fetching web results of a search query from Google Search Engine into the excel file. User can enter any search query and select the number of URLs to be assessed on the basis of trust value. It is observed that results returned by Google search engine for a search query is repetitive in nature. The web-links or URLs produced by the search engine points to the different web pages of the same website. To undermine this, unique websites are identified with their frequencies. Further Similarweb APIs are used to collect identified web metrics for all these unique websites for the desired time period. These APIs re provided by Similarweb.com which "provides services in business intelligence, data mining and web analytics for international corporations. It uses big data techniques to analyze and corroborate data at the world level" [32]. Figure 2 shows the interface of the TNM tool. JAVA is used for API programming as it is robust and flexible in nature. Moreover, its multi-threading feature helps in reducing the response time.

Figure 2: Interface of TNM tool

The request to obtain the metrics is made using API calls as shown. The result is returned in JSON (JavaScript Object Notation) format. It is parsed to get the values of desired metrics. [URL] is replaced by the actual URL of the website for which the data needs to be fetched and [USERKEY] is replaced by the developer key generated from Similarweb.com. Average Time on Website:

http://api.similarweb.com/Site/[URL]/v1/visitduration?gr=daily&start=M-YYYY&end=M-YYYY&md=true& Format=JSON&UserKey=[USERKEY]

Bounce Rate

http://api.similarweb.com/Site/[URL]/v1/bouncerate?gr=daily&start=M-YYYY&end=M-YYYY&md\=true& Format=JSON&UserKey=[USERKEY]

Average Daily Visits

http://api.similarweb.com/Site/[URL]/v1/visits?gr=daily&start=M-YYYY&end=M-YYYY&md=true&Format =JSON&UserKey=[USERKEY]

Pages/Visit

http://api.similarweb.com/Site/[URL]/v1/pageviews?gr=daily&start=M-YYYY&end=M-YYYY&md=true& Format=JSON&UserKey=[USERKEY]

3.2. Data Preprocessing and Implementation

The data collected via TNM tool's interface is preprocessed and normalized using R. "R is an open source software environment which supports strong statistical computations and graphics. It supports UNIX, Mac OS and Windows" [33]. Moreover, it has an API for interfacing with JAVA. The collected data is normalized to ensure that "attributes with large domains does not dominate attributes with smaller domains and avoids the dependence on the choice of measurement units (also referred as standardization of data). It helps in giving all attributes an equal weight" [34].The data is normalized for all the metrics. Categorical ranking (fetched through Similarweb API) is labeled using data discretization (or concept hierarchy generation) by binning method. Three bins are used for categorizing as found in a direct study that five or more levels of trust instead of three becomes "more challenging for individuals to perceive" [35].The same is supported by [36].

Multiple Discriminant Analysis technique is used to obtain the trustworthiness of websites returned by Google search engine. This technique gives the loading factors or weights for each of the metrics thus forming an equation for calculating trust. On the basis of the values returned by the equation, website is categorized into one of the three levels of trust.

4. Empirical Evaluation

Evaluation of TNM tool was done by carrying out an experiment with search query 'balanced diet' and validating the rankings obtained with ratings obtained by Web of Trust and WebCast Tool. Web of Trust (WOT) is "based on crowd sourcing approach that collects ratings and reviews from a global community of millions of users who rate and comment on websites based on their personal experiences" [37], whereas WebCast[21] is a credibility assessment tool conceptualized and implemented to arrange websites according to the order of their credibility. The latter has been evaluated empirically with the help of human judges (especially students) and has emerged as a promising tool for assessing web information credibility. If the results of TNM tool are positively correlated with results obtained from WOT and WebCast, then the persistence of TNM tool is also established.

In order to carry out this evaluation, Google key, search keywords 'balanced diet' and number of URLs as 500 were entered to the TNM tool. It returned top 500 URLs for it in an excel sheet. This excel sheet served as input to the second part of the interface where Similarweb key was entered, start date as 01-2015 and end date as 07-2015 (January, 2015 to July, 2015). The dataset returned was then analyzed with R programming. As stated in previous sections, number of unique websites is much less (53) than the number of websites fetched (500). R returned the websites with their calculated trust categories._

RRCcniol« o j E

3 -C.1008570 0.1931637 -0.1393479 -0.1591155

Cct:iicier.;a ot linear diacrurj.nar.ca:

LD1 LD2

Pace.Visit -0.6403273 -C.6558052

;Bounce. Rate -0.2530642 C.4719802

Averaae.Viait.Duration -1.1974528 1.0469C02 Averaoe.Dally.Visits 0.7333339 -0.6907542

Proportion o£ trace:

LD1 LD2 0.8286 0.1714 |> lda_pred-predict<lda_»odel,x>

> r.a-»3 (iaa_Freai

IX1 "class" "pcsterior" "x"

> Iris prrrt Siiilr-Irtn privlfirlB^^

> cable(lfla_jrr<1_Bnr.k, xfVS)

Ida pred Far.* 12 3

1 22 3 7

2 10 0 3 6 1 14

> seaa | lda_pred_Rar.lt: "x$V5) |[1] 0.3333333

Figure 3: Results for 'balanced diet'

LD1 and LD2 represents the two discriminant functions for three trust categories (1 represents most trustworthy and 3 represents least trustworthy). Two functions are obtained as number of discriminant functions is always one

less than the number of classes or categories. Both LD1 and LD2, as shown in Figure 3 represent Discriminant Loadings for all the four web metrics chosen as antecedents of Trust. Proportion of trace is the proportion of between class variance that is explained by successive discriminant functions. Classification matrix was obtained by applying these Discriminant functions to test the performance of this technique and misclassification error obtained was 33.33%. Classification matrix can also be considered as error matrix. It is a "specific table layout that allows visualization of the performance of an algorithm and contains information about actual and predicted classifications" [38]. Ideally off-diagonal elements of the matrix should be zero as diagonal elements depict the number of correct classifications and off-diagonal elements depict erroneous classifications.

This same experiment was run using WOT and WebCast as discussed earlier. We achieved a positive correlation with both the tools having values 0.87 and 0.71 respectively. The results are encouraging and illustrate TNM as an acceptable tool.

Conclusion

Websites are made by humans and used by humans. Therefore, behaviour or acceptance of humans should be considered as a qualifying criteria for judging the credibility of a website. A methodical approach is used to model this behaviour to quantify trust value of a website. Though the study has been done only in the dominion of informational websites, it could further be enhanced for websites including audios and videos. The authors adopted a completely novel approach for calculating trust from analytical data, as this has never been explored beyond a point for trust modelling to the best of knowledge.

This work provides an initiating platform for further exploration of quantifying trust of information-led or content-driven websites. Prior comprehensive aspects of trust may be incorporated in the model to further improve it. Unsupervised learning such as cluster analysis can also be used in place of supervised learning used for finding initial trust labels. Moreover, clustering can also be used while pre-processing the data to further prune the outliers. Instead of LDA (Linear Discriminant Analysis), QDA (Quadratic Discriminant Analysis) can also be used by readjusting the bin size for trust categorization. Additionally, bin size can be made dynamic for finding relative trust levels among the websites.

References

1. Gundecha P., Ranganath, S., Feng, Z., Liu, H. A tool for collecting Provenance Data in Social Media. KDD'13, August 11-14, 2013, Chicago, Illionois, USA.

2. Miriam J. Metzger. Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Wiley InterScience. Published online: 21 September, 2007 (www.interscience.wiley.com)

3. Eysenbach, G., Powell, J., Kuss, O., Sa, E. R. (2002). Empirical Studies assessing the quality of health information for consumers on the World Wide Web: A systematic review. Journal off the American Medical Association, 287(20), 2691-2700.

4. Knust, H., Groot, D., Latthe, P.M., Latthe, M., Khan, K.S. (2002). Accuracy of Information on apparently credible websites: Survey of five common health topics. British Medical Journal, 324, 581-582.

5. Morahan-Martin, J.M., Anderson, C. (2000). Information and misinformation online: Recommendations for facilitating accurate mental health information retrieval and evaluation. CyberPsychology& Behavior, 3, 731-742.

6. Rice, R.E., Katz, J.E. (2001). The internet and health communication: Experiences and Expectations. Thousand Oaks, CA: Sage.

7. Seidman, J.J. (2006). The mysterious maze of the World Wide Web: How can we guide the consumers to high-quality health information on the Internet? In M. Murero& R.E. Rice (Eds.), Internet and health care: Theory, research and practice (pp. 195-212). Mahwah, NJ:Erlbaum.

8. Freeman K. S., Spyridakis, J.H. (2004). An examination of factors that affect the credibility of online health information. Technical Communication. 51(2), 239-263.

9. Corritore, C.L., Kracher, B., Weidenbeck, S. (2003). Online trust: concepts, evolving themes, a model. International Journal of HumanComputer Studies 58, no. 6, 737-758.

10. Lucassen, T., Muilwijk, R., Noordjiz, M. L., Schraagen, J.M. (2013). Topic familiarity and information skills in online credibility evaluation. journal of the American society for Information Science and Technology 64 (2), 254-264.

11. Lucassen, T., Schraagen, J.M. (2012). The role of topic familiarity in online credibility evaluation support. Human factors and Ergonomics society. 56(1), SAGE publications, 1233-1237.

12. Lucassen, T., Schraagen, J.M. (2013). The influence of source cues and topic familiarity on credibility evaluation. Computers in Human Behavior, 29(4), 1387-1392.

13. Lucassen, T., Schraagen, J.M. (2010). Trust in Wikipedia: how users trust information from an unknown source. 4th Workshop on Information credibility, ACM Press, 19-26.

14. Evaluating Internet Information. (2015). Online learning Library center. http://www.usg.edu/galileo/skills/unit07/internet07_08.phtml.

(accessed on 06/09/2015).

15. Fogg, B.J., Tseng, H. (1999) The elements of computer credibility. SIGCHI conference on Human factors in computing systems, ACM Press, 80-87.

16. Grassian, (1995). E. Thinking critically about World Wide Web resources. UCLA Library http://www2.library.ucla.edu/libraries/college/11605_12337.cfm (accessed on 10/07/2015).

17. Beck, S. (1997). The Good, The Bad &The Ugly: or, why it's a good idea to evaluate web resources. http://lib.nmsu.edu/instruction/evalcrit.html (accessed on 15/07/2015).

18. UC Berkeley. (2010). Teaching Library Internet Workshops, Finding Information on the Internet: A tutorial. http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Evaluate.html (accessed on 10/07/2015).

19. Pattanaphanchai, J., O'Hara, K., Hall, W. (2013) Trustworthiness criteria for supporting users to assess the credibility of w eb information. 22nd International conference on World Wide Web companion.

20. Aggarwal, S.,Oostendorp, H.V. (2011). An attempt to automate the process of source evaluation. ACEEE International Journal on Communication 2(2).

21. Aggarwal, S., Oostendorp, H.V., Reddy, Y.R.B., Indurkhya, B. (2014) Providing Web Credibility Assessment Support. European Conference on Cognitive Ergonomics ECCE'14.

22. Amin, A., Zhang, J., Cramer, H., Hardman, L., Evers, V. (2009) The effects of source credibility ratings in a cultural heritage information aggregator. 3rd workshop on Information credibility on the web, ACM Press, 35-42.

23. Sondhi, P., Vydiswaram, V.V., Zhai, C. (2012). Reliability prediction of webpages in the medical domain. Advances in Information retrieval, Springer Berlin Heidelberg, 219-231.

24. Olteanu, A., peshterliev, S., Liu, X., Aberer, K.(2013) Web credibility: Features exploration and credibility prediction. Advances in Information Retrieval, Springer Berlin Heidelberg, 557-568.

25. Ennals, R., Byler, D., Agosta, J.M., Rosario, B. (2010). What is disputed on the web? Workshop on Information credibility, ACM Press, 6774.

26. Dawra, V. (2015). Whom can you trust? Times Life: rev up your life! http://timesofindia.indiatimes.com/ accessed on 9/09/2015.

27. Kaushik, A. (2015). Standard Metrics Revisited: #3: Bounce Rate. Occam's Razor by Avinash Kaushik. http://www.kaushik.net/avinash/standard-metrics-revisited-3-bounce-rate/ (accessed on 09/09/2015).

28. Singal, H., Kohli, S. (2014). Conceptual model for obfuscated trust induced from Web Analytics data for content-driven websites. Proceedings of ICACCI IEEE 2014.

29. Kohli S., Singal, H. (2015). A methodological approach for Quantizing Trust from Human Behavior for Content-Driven Websites. In Proceedings of Special Issue, Cyber Security, Privacy and Forensics for Complex Systems, IJSWCC, Bentham Publishers, 2015.

30. Singal, H., Kohli, S. (2015). Escalation of Trust Analysis in Web. In Proceedings of the 12th ACM International Conference on Computing Frontiers.

31. Custom Search. (2015). https://developers.google.com/custom-search/ (accessed on 10/05/2015).

32. SimilarWeb. (2015). https://developer.similarweb.com/ (accessed on 09/04/2015).

33. R Core Team (2014). "R: A language and environment for statistical computing", R Foundation for Statistical Computing, Vienna, Austria, http : //www.R-proj e ct. org/.

34. Han J., Kamber M., Pei J. (2012). Data Mining: Concepts and Techniques. 3rd Edition, Morgan Kaufmann, Elsevier.

35. Nurse JRC, Agrafiotis I., Goldsmith M., Creese S., Lamberts K. (2014). Two sides of the coin: measuring and communicating the trustworthiness of online information. Journal of Trust Management, Springer.

36. Idris NH, Jackson MJ, Abrahart RJ. (2011). Colour coded traffic light labeling: A visual quality indicator to communicate credibility in map mash-up applications. International Conference on Humanities Social Sciences, Science &Technology.

37. Web of Trust. (2015). www.mywot.com (accessed on 08/09/2015).

38. Confusion Matrix. (2015). http://en.wikipedia.org/wiki/Confusion_matrix (accessed on 29/04/2015).