Deep Web Research and Discovery Resources 2019

How big is the Deep Web? It is estimated that the Deep Web comprises 7,500 terabytes – although an exact size is not known, and figures vary widely on this question. The magnitude, complexity and siloed nature of the Deep Web means researchers cannot turn to one specific guide or one search engine to access the vast range of information, data, files and communications that comprise it. The ubiquitous search engines (Google, Bing, Yahoo) manage, index and deliver results from the Surface web. These search results include links, data, information, reports, news, subject matter content and a large volume of advertising that is optimized to increase traffic to specific sites and support marketing and revenue focused objectives. On the other hand, the Deep Web – which is often misconstrued as a repository of dark and disreputable information [Note – it is not the Dark Web], has grown tremendously beyond that characterization to include significant content on a wide range of subject matter covering a broad swath of files and formats, databases, pay-walled content as well as communications and web traffic that is not otherwise accessible through the surface Web.

In the last several years many more applications, tools and techniques have become available that provide researchers with targeted access to specific portions of the Deep Web using discovery techniques that leverage algorithms to access data and information in many formats. Documents used by organizations and entities to communicate within their domains, including databases and files of information that may be password protected and encrypted communications, may also be accessible. Searching for this information requires that users apply search engines, techniques and search tools that function as a means of accessing these repositories of corporate, data, news and research that is not transparent using the Surface web activities. Searching the Deep Web requires the use of multiple techniques and resources – it is challenging, complicated and far less straight forward than just using Google, and much more time consuming as well. This guide providers researchers with an extensive range of sites, services, resources, tools and techniques from which to choose so as to target their efforts to locate and then utilize information and data from within the Deep Web. In addition, this guide assists researchers to identify and implement security and privacy applications while using the Deep Web, as well as offering a large number of training and learning resources to conduct more effective and efficient searches.

This Deep Web Research and Discovery Resources 2019 report and guide is divided into the following topics:

Articles, Papers, Forums, Audios and Videos
Cross Database Search Services
Cross Database Search Tools
Peer to Peer, File Sharing, Grid/Matrix Search Engines
Presentations
Resources – Deep Web Research
Resources – Semantic Web Research
Bot and Intelligent Agent Research Resources and Sites

Articles, Papers, Forums, Audios and Videos

8 Best Deep Web People Search Engines
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/16323-8-best-deep-web-people-search-engines-updated

10 Things You Should Know About Dark Web Websites by Josh Fruhlinger – November 23, 2018
https://www.csoonline.com/article/3322134/cyber-attacks-espionage/10-things-you-should-know-about-dark-web-websites.html

99 Resources to Research & Mine the Invisible Web by Jessica Hupp
https://www.collegedegree.com/library/college-life/99-resources-to/

A Basic Guide to Diving In To the Dark Web
https://www.zdnet.com/pictures/a-basic-guide-to-the-deep-dark-web/

A Beginner’s Guide to the Dark Web by Ben Dickson
https://www.dailydot.com/layer8/what-is-dark-web/

Academic and Scholar Search Engines and Sources 2019
https://www.ScholarSearchEngines.com/

Accessing the Deep Web and Dark Web with Tor by Jack Jones
https://mobile.audible.com/search.htm?narrator=Dan+Gralick

Access the Deep Web and Protect Your Privacy Online with Anonabox by Marco Chiappetta
https://www.forbes.com/sites/marcochiappetta/2016/04/29/access-the-deep-web-and-protect-your-privacy-online-with-the-anonabox/#72cd0dc337c2

All of OCLC’s WorldCat Heading Toward the Open Web by Barbara Quint
https://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16353

An Executive Overview of the Deep and Dark Web
https://www.cio.com/article/3199632/cyber-attacks-espionage/an-executive-overview-of-the-deep-dark-web.html

An Insider’s Look Into the Dark Web by Aaron Tan
https://www.computerweekly.com/feature/An-insiders-look-into-the-dark-web

An Interactive Clustering-based Approach to Integrating Source Query interfaces on the Deep Web by W. Wu, C. Yu, A. Doan, W. Meng
https://www.cs.binghamton.edu/~meng/pub.d/sigmod04-final.pdf

Annotation for the Deep Web
https://dl.acm.org/citation.cfm?id=1137372

An Up-To-Date Layman’s Guide To Accessing The Deep Web
https://www.fastcompany.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web

A Primer on Staying Secure and Anonymous on the Dark Web by Mark Turner
https://www.techspot.com/guides/1292-web-security-anonymizer-primer/

Articles on Deep and Dark Web Published by Associations of Internet Research Specialists (AIRS)
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/itemlist/category/20-deep-web

Automatic Extraction of Web Search Interfaces for Interface Schema Integration by H. He, W. Meng, C. Yu, Z. Wu
https://www.cs.binghamton.edu/~meng/pub.d/WWWposterhe.pdf

Automatic Information Extraction From Semi-Structured Web Pages By Pattern Discovery
https://dl.acm.org/citation.cfm?id=640423&dl=ACM&coll=portal

Automatic Meaning Discovery Using Google by Rudi Cilibrasi and Paul M. B. Vitanyi
https://arxiv.org/abs/cs.CL/0412098

Battle of the Secure Smartphones
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17276-battle-of-the-secure-smartphones

Bibliomining for Automated Collection Development in a Digital Library Setting: Using Data Mining to Discover Web-Based Scholarly Research Works by Dr. Scott Nicholson
https://scottnicholson.com/pubs/asisdiss.pdf

Big Data and AI: 30 Amazing (And Free) Public Data Sources for 2018 by Bernard Marr
https://www.forbes.com/sites/bernardmarr/2018/02/26/big-data-and-ai-30-amazing-and-free-public-data-sources-for-2018/

Bot Research 2019
https://www.BotResearch.info/

BrightPlanet Launches Deep Web Data Feeds: Global News Data Feed Is First Available Data Feed
https://www.brightplanet.com/2015/01/brightplanet-launches-deep-web-data-feeds-global-news-data-feed-first-available-data-feed/

Client-Side Deep Web Data Extraction
https://www.computer.org/csdl/proceedings/cec-east/2004/2206/00/22060158-abs.html

Clustering E-Commerce Search Engines by Q. Peng, W. Meng, H. He, C. Yu
https://www.cs.binghamton.edu/~meng/pub.d/WWWposterPeng.pdf

Common Deep Web and Big Data Questions Answered (Part 1)
https://www.brightplanet.com/2014/11/common-deep-web-big-data-questions-answered-part-1/

Common Deep Web and Big Data Questions Answered (Part 2)
https://www.brightplanet.com/2014/12/common-deep-web-big-data-questions-answered-part-2/

Creating Intelligence from Big Data
https://bigdata.brightplanet.com/creating-new-intelligence-from-big-data

Current Awareness Discovery Tools on the Internet 2019
https://www.zillman.us/white-papers/current-awareness-discovery-tools-on-the-internet/

Current Awareness Tools 2019
https://www.currentawarenesstools.com/

DarkCyber Beyond Search by Stephen E. Arnold
https://arnoldit.com/wordpress/category/dark-web/

Darknet Resembles the Open Internet – Just With Anonymity by Roderick S. Graham and Brian Pitman
https://www.newsweek.com/darknet-resembles-internet-itself-anonymity-550324?rx=us

DarkReading
https://www.darkreading.com/

Dark Web (DarkNet) by Margaret Rouse
https://whatis.techtarget.com/definition/dark-web

Dark Web Data Theft May Have Finally Met Its Match by Vaughn Highfield
https://www.alphr.com/security/1009340/dark-web-data-theft-fujitsu-intsights

Darkweb for Newbees: Hacking and Marketplaces by Gaurav Sangwani
https://www.businessinsider.in/darkweb-for-newbees-hacking-and-marketplaces/articleshow/55941181.cms

Dark Web News
https://www.DarkWebNews.com/

Dark Web Security Tips by Enjeck Mbeh Cleopatra
https://latesthackingnews.com/2018/07/17/dark-web-security-tips/

Dark Web Version of Facebook Shows a New Way to Secure the Web by Tom Simonite
https://www.technologyreview.com/s/532256/dark-web-version-of-facebook-shows-a-new-way-to-secure-the-web/

Dark Web – CRS Report by Kristin Finklea
https://fas.org/sgp/crs/misc/R44101.pdf

Deep Web by Margaret Rouse
https://whatis.techtarget.com/definition/deep-Web

Deep Web – Exploring the Secrets of the Hidden Internet by Marcus P. Zillman, M.S., A.M.H.A., – 23 minutes – Internet/Technology Channel
https://www.planetearthradio.com/technology.htm

Deep Web: Legal Due Diligence by Lisa Brownlee
https://federatedsearchblog.com/2015/04/29/deep-web-legal-due-diligence/

Deep Web Links – Massive Deep Websites Links and URL by Martin Grossner
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/19104-deep-web-links-massive-deep-websites-links-and-url

Deep Web May Be 500 Times Bigger Than Normal Web
https://mobile.wnd.com/2018/09/deep-web-may-be-500-times-bigger-than-normal-web/

Deep Web Research and Discovery Resources 2019
https://www.DeepWeb.us/

Deep Web Search Engines to Explore the Hidden Internet by Rakesh Krishnan
https://cdn.ampproject.org/c/s/amp.thehackernews.com/thn/2016/02/deep-web-search-engine.html

Deep Web Technologies Launches Explorit! EBM, the leading Evidence-Based Medicine Search Engine for Medical Libraries
https://www.benzinga.com/pressreleases/18/06/r11847737/deep-web-technologies-launches-explorit-ebm-the-leading-evidence-based

Deep Web: The Mysterious Side of the Internet and How To Access It by Michael Lowry
https://factschronicle.com/deep-web-the-mysterious-side-of-the-internet-and-how-to-access-it-14770.html

Deep Web: What Is It and How To Access it? – The Ultimate Guide by Samantha Carter
https://darkwebnews.com/deep-web/

Digging Deeper into Deep Web Databases by Breaking Through the Top-k Barrier
https://arxiv.org/abs/1208.3876

Digital Shadows Launches Shadow Search
https://www.darkreading.com/threat-intelligence/digital-shadows-launches-shadow-search/d/d-id/1332921

Digital Shadows Launches Shadow Search – Enabling Organizations to Self-Serve and Find Cyber Threats Against Them from Across The Open, Deep and Dark Web
https://www.businesswire.com/news/home/20180926005135/en/Digital-Shadows-Launches-Shadow-Search%E2%84%A2-%E2%80%93-Enabling

Diving Into the Darknet by Michal Salat
https://blog.avast.com/diving-into-the-darknet

Do’s and Don’ts of the Dark Web by Martin Beltov
https://securityboulevard.com/2018/07/dos-and-donts-of-the-dark-web/

Effective and Scalable Metasearch Project
https://www.cs.binghamton.edu/~meng/metasearch.html

Efficient Deep Web Crawling Using Reinforcement Learning
https://link.springer.com/chapter/10.1007%2F978-3-642-13657-3_46

Everything You Need To Know About the Deep Web In One Simple Infographic
https://www.businessinsider.com/everything-you-need-to-know-about-the-deep-web-in-one-simple-infographic-2015-2

Experiences In Crawling Deep Web In The Context Of Local Search
https://dl.acm.org/citation.cfm?id=1460016

First Comprehensive Map of the Dark Web Reveals a Remarkable Antisocial Corner of the Internet by Matthew Hutson
https://www.sciencemag.org/news/2017/04/first-comprehensive-map-dark-web-reveals-remarkably-antisocial-corner-internet

Grey Literature
https://en.wikipedia.org/wiki/Gray_literature

Grey Literature Network Service (GreyNet)
https://www.greynet.org/

Hackers Aggregating Data Leaks on the Dark Web
https://it.toolbox.com/article/hackers-aggregating-data-leaks-on-the-dark-web

Here Are the 10 Best Deep Web Search Engines by Kristen Hubby
https://www.dailydot.com/layer8/best-deep-web-search-engines/

How Does the Tor Browser Functions? By Ben Prince
https://www.technologynewsextra.com/how-does-the-tor-browser-functions/96041.html

How Scary Is the Dark Web by Aria Inthavong
https://www.buzzfeed.com/watch/video/68371

How To Access the Dark Web: What Is Tor and How Do I Access Dark Websites? By Victoria Woollaston
https://www.alphr.com/technology/1002667/how-to-access-the-dark-web-what-is-tor-and-how-do-i-use-it

How To Access the Dark Web by Brandt Ranj
https://www.independent.co.uk/life-style/gadgets-and-tech/how-to-access-the-dark-web-a7047041.html

How To Access the Dark Web by Tyler Lacoma
https://www.digitaltrends.com/computing/how-to-access-the-dark-web/

How to Access the Dark Web (Deep Web)
https://www.thedarkweblinks.com/how-to-access-the-dark-web/

How To Access the Dark Web Using Tor by Cozmo
https://mixance.com/how-to-access-the-dark-web-using-tor/

How To Access the Deep Web Anonymously by Ritika Tiwari
https://www.cloudwards.net/the-deep-web/

How To Access the Hidden Wiki?
https://talkdailynews.com/how-to-access-the-hidden-wiki/

How To Browse the Deep Web Using Tor on iPhone and iPad by Sumeet Sharma
https://www.iappnalysis.com/2015/04/browse-deep-web-on-iphone-ipad-using-tor-browser.html

How To Do a Deep Web Search and Why You Should by Kelli Uhrich of Komando.com
https://www.komando.com/tips/396770/how-to-do-a-deep-web-search-and-why-you-should

How to Mine the Invisible Wev: The Ultimate Guide by Jerri Collins
https://www.lifewire.com/mine-invisible-web-3482495

How To Safely Access The Dark Web by David Nield
https://www.lifehacker.com.au/2018/03/how-to-search-the-deep-web-safely/

How To Tread Lightly Into The Dark Web by Logan Hochstetler
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18649-how-to-tread-lightly-into-the-dark-web

How To Use a VPN with Tor
https://www.expressvpn.com/how-to-use-vpn/tor-vpn

If Your Data Is Found On the Dark Web, Firefox Monitor Will Let You Know by Chuong Nyguyen
https://www.digitaltrends.com/computing/firefox-monitor-lets-your-know-if-youve-been-pwned/

Illuminating the Dark Web by Robert Gehl
https://www.scientificamerican.com/article/illuminating-the-dark-web

Inequality and the Intellectual Dark Web by Christian Alejandro Gonzalez
https://www.nationalreview.com/2018/05/intellectual-dark-web-members-united-views-inequality/

Information Retrieval and the Semantic Web by Tim Finin, James Mayfield, Clay Fink, Anupam Joshi, and R. Scott Cost
https://ebiquity.umbc.edu/paper/html/id/185/

In Search of the Deep Web
https://www.salon.com/2004/03/09/deep_web/

Inside the Dark Web: The Truth Is There Is a Lot of Evil Out There by Conor Pope
https://www.irishtimes.com/culture/tv-radio-web/inside-the-dark-web-the-truth-is-there-is-a-lot-of-evil-out-there-1.3653092

Intellectual Dark Web: New Movement or Just a Rebarnding of Old Ideas? By Kevin Dickinson
https://bigthink.com/kevin-dickinson/what-is-the-so-called-intellectual-dark-web.amp

IR and IE on the Web – PhD and MSc Dissertations
https://groups.yahoo.com/neo/groups/webir/info
https://www.webir.org/

Is the Tor Browser Safe? Why Should You Use It? By Mohammed Mujtab
https://technocodex.com/is-tor-browser-safe-why-should-you-use-it/

Journey Into the Dark by Jason Smith
https://medium.com/@jasisrad/journey-into-the-dark-8c7922a48265

Journey Into the Hidden Web: A Guide for New Researchers by Ryan Dube
https://www.makeuseof.com/tag/journey-into-the-hidden-web-a-guide-for-new-researchers/

Just the Tip of the Iceberg: Why You Should Be Monitoring the Deep Web
https://www.information-age.com/technology/security/123461668/just-tip-iceberg-why-you-should-be-monitoring-deep-web

Lessons from the Deep Web That Could Lead To a More Secure IoT by Revathl Subramanian
https://blogs.ca.com/2015/04/02/lessons-from-the-deep-web-that-could-lead-to-a-more-secure-iot/?mrm=425878&cid=GLOB-SMM-ABUS-AAR-000002-00000571

Mining Newsgroups Using Networks Arising From Social Behavior
https://dl.acm.org/citation.cfm?id=775227

Mining the Deep Web: Search Strategies That Work by Lee Ratzan
https://www.computerworld.com/s/article/9005757/Mining_the_Deep_Web_Search_strategies_that_work?pageNumber=1

Mining Topic-Specific Concepts and Definitions on the Web
https://www.cs.uic.edu/~liub/publications/WWW-2003.pdf

NASA Is Indexing the Deep Web to Show Mankind What Google Won’t by Danielle Bronner
https://fusion.net/story/145885/nasa-is-indexing-the-deep-web-to-show-mankind-what-google-wont/

New AI System DARKMENTION Will Detect Upcoming Cyberattackes From Dark Web by Waqas
https://www.hackread.com/ai-system-darkmention-detects-cyberattacks-from-dark-web

Next Generation TOR is Arriving and It Will Include a Distributed Random Number Generator
https://securityaffairs.co/wordpress/47683/deep-web/distributed-tor-random-number-generator.html

NYU-Poly Researcher Awarded DARPA Contract To Explore the Deep Web by Rhea Kelly
https://campustechnology.com/articles/2015/02/25/nyu-poly-researchers-awarded-darpa-contract-to-explore-the-deep-web.aspx

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/

OnionLand Demystifying the Mystery of the Deep Web
https://m.digitaljournal.com/pr/3927243

OntoMiner: Bootstrapping and Populating Ontologies From Domain Specific Web Sites
https://www.public.asu.edu/~hdavulcu/VLDB-WS03.pdf

OpenIndex – Creating a Public Internet Index
https://www.openindex.org

Out-googling Google: Federated Searching and the Single Search Box
https://library.marist.edu/ACRL/Foxhunt_demo.html

Privacy, the Dark Web and Hacker Devices by Don Hawkins
https://www.libconf.com/2018/04/18/privacy-the-dark-web-and-hacker-devices/

Publications about Web Analysis, Web Search, Citation Indexing, Digital Libraries, Machine Learning, Neural Networks
https://research.google.com/

QProber: Classifying and Searching “Hidden-Web” Text Databases
https://www.ipeirotis.com/?project=qprober-classifying-and-searching-hidden-web-text-databases

Really Private Browsing: An Unofficial User’sGuide to Tor by Andre Infante
https://www.makeuseof.com/tag/really-private-browsing-an-unofficial-users-guide-to-tor/

Research Beyond Google: 56 Authoritative, Invisible, and Comprehensive Resources
https://oedb.org/ilibrarian/research-beyond-google/

Researchers Call for a Shared Dark Web Taxonomy by Phil Muncaster
https://www.infosecurity-magazine.com/news/researchers-call-shared-dark-web

Scan the Dark Web For Threat Intelligence by Michelle Drolet
https://www.csoonline.com/article/3245587/cyber-attacks-espionage/scan-the-dark-web-for-threat-intelligence.html

Scientific American: Featured Article: The Semantic Web In Action
https://www.scientificamerican.com/article/semantic-web-in-actio/

Search Engine Meeting Archives
https://www.SearchEngineMeeting.net/

Search Engine Technology and Digital Libraries
https://www.dlib.org/dlib/june04/lossau/06lossau.html

Searching the Dark Web for Business Intelligence by Robert Vamosi
https://www.forbes.com/sites/robertvamosi/2018/04/28/searching-the-dark-web-for-business-intelligence/

Searching the Deep Web by Alex Wright
https://mags.acm.org/communications/200810/?pg=16

Searching the Deep Web
https://www.dlib.org/dlib/january01/warnick/01warnick.html

Searching the Internet 2018 – The Primer  – 2019 Guide to Searching the Internet
https://www.SearchingTheInternet.info/

Search Interfaces on the Web: Querying and Characterizing by Denis Shestakov
https://www.scribd.com/document/231936534/Search-Interfaces-on-the-Web-Querying-and-Characterizing

Seeing through the ‘invisible’ Web
https://usatoday30.usatoday.com/tech/2001/10/15/invisible-web-search.htm

Semantic Web Content Accessibility Guidelines for Current Research Information Systems (CRIS) by A. Lopatenko
https://derpi.tuwien.ac.at/~andrei/AURIS_DE.htm

Shining a Light on the So-Called Dark Web by Kate Knibbs
https://www.theringer.com/tech/2018/5/10/17339900/dark-web-lexicon

Startup CYRCON’s Artificial Intelligence Predicts Enterprise Cyberattacks by PRWEB, November 5, 2018
https://www.prweb.com/releases/startup_cyr3cons_artificial_intelligence_predicts_enterprise_cyberattacks/prweb15887247.htm

Staying Safe in the Shadow of the Dark Web
https://www.benzinga.com/pressreleases/18/05/p11804571/staying-safe-in-the-shadow-of-the-dark-web

Stop Talking About the Dark Web: Tor Project CoFounder Roger Dingledine
https://technical.ly/philly/2017/05/15/dark-web-roger-dingledine/

Stop Worrying About the Dark Web by Rich Matta
https://www.forbes.com/sites/forbestechcouncil/2018/10/15/stop-worrying-about-the-dark-web/#31e060935fcc

Structured Databases on the Web: Observations and Implications
https://dl.acm.org/citation.cfm?id=1031584

Testbed for Information Extraction from Deep Web
https://research.microsoft.com/users/nickcr/pubs/yamada_www2004poster.pdf

The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17217-top-10-best-deep-web-search-engines-to-explore-hidden-web

The 10 Best Deep Web Search Engines to Explore Hidden Web by Issac Avila
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18749-top-10-best-deep-web-search-engines-to-explore-hidden-web

The Black Market Report
https://www.armor.com/vantage-points/black-market-report/

The Darknet: A Quick Introduction for Business Leaders by Trey Degrassi
https://hbr.org/2016/12/the-darknet-a-quick-introduction-for-business-leaders

The Dark Web and How To Access It by MacKenzie Sigalos
https://www.cnbc.com/2018/04/13/the-dark-web-and-how-to-access-it.html

The Dark Web and Remote Communication – November 10, 2018 – Tech Talk Radio
http://federalnewsnetwork.com/stratford-universitys-tech-talk/2018/11/the-dark-web-and-remote-communication-november-10-2018/

The Dark Web: What Every Parent Should Know by Cybermum Australia
https://securingtomorrow.mcafee.com/consumer/family-safety/dark-web-what-every-parent-should-know/

The Dark Web What It Is and Staying Safe by Annie Quireshi
https://sustainablebusinessforum.com/dark-web-what-it-is-and-staying-safe

The Dark Web? Why Small Businesses Should Concern Themselves With the Threat by Jim Anderson
https://www.entrepreneur.com/article/320347

The Deep Web May Be 500 Times Bigger Than the Normal Web
https://www.cnbc.com/2018/09/06/beyond-the-valley-understanding-the-mysteries-of-the-dark-web.html

The Deep Web: Surfacing Hidden Value by Michael K. Bergman
https://quod.lib.umich.edu/j/jep/3336451.0007.104?view=text;rgn=main

The Deep Web: What’s Lurking in the Underbelly of the Internet? By Michelle Alvarez
https://securityintelligence.com/the-deep-web-whats-lurking-in-the-underbelly-of-the-internet/#.VUIUNmfD9D8 

The Future Of News: The Digital Information Librarian
http://www.masternewmedia.org/2004/03/24/the_future_of_news_the.htm

The Growing Force That Will Soon Reshape The Entire Internet by Frank Minite
https://www.forbes.com/sites/frankminiter/2017/12/30/the-growing-force-that-will-soon-reshape-the-entire-internet/

The Hidden Potential of the Web
https://www.theguardian.com/society/2004/apr/21/epublic.technology18

The Intellectual Dark Web Conservatives Fear by Taylor Link
https://www.salon.com/2018/09/02/the-intellectual-dark-web-conservatives-fear

The Invisible Web by Chris Sherman
https://web.freepint.com/go/newsletter/64#feature

The Invisible Web: What it is, Why it exists, How to find it, and Its Inherent Ambiguity
https://www.newworldencyclopedia.org/entry/Deep_Web

The Invisible Web: Where Search Engines Fear To Go
https://www.powerhomebiz.com/vol25/invisible.htm

The Most Amazing Deep Web Search Engines Finally Introduced To the World Called: OnlindLandSearchEngines.com .
https://m.digitaljournal.com/pr/3928224

The New Search Engines Shining a Light On the Deep Web by Carola Frediani
https://kernelmag.dailydot.com/issue-sections/features-issue-sections/10376/how-to-search-deep-web-tor/

The Other Side of Internet by Robert Johnny, Jr.
https://it.toolbox.com/blogs/robertjohnnyjr/the-other-side-of-internet-090618

The Problems With Searching the Deep Web by Nancy K. Herther
https://newsbreaks.infotoday.com/NewsBreaks/The-Problems-With-Searching-the-Deep-Web-121884.asp

The Super-Private Tor Browser Get a Huge Update, But Should You Switch From Chrome?
https://gizmodo.com/the-super-private-tor-browser-gets-a-huge-update-but-s-1828879886

The Ultimate Guide to the Invisible Web
https://oedb.org/ilibrarian/invisible-web/

The Virtual Private Library™ and The Deep Web Video by Melissa Barker
https://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html

Things You Can Do On the Dark Web That Aren’t Illegal by Jacob Kleinman
https://lifehacker.com/things-you-can-do-on-the-dark-web-that-arent-illegal-1819790298

This Tool Lets You Scan the Dark Web for Your (stolen) Personal Data by Anna K. Sasaki
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17251-this-tool-lets-you-scan-the-dark-web-for-your-stolen-personal-data

Timeline of Events Related to the Deep Web
https://papergirls.wordpress.com/2008/10/07/timeline-deep-web/

Top 5 Legal Reasons To Use the Darknet by JP Buntinx
https://themerkle.com/top-5-legal-reasons-to-use-the-darknet/

Topological Measures and Maps Of the Web
https://informatics.indiana.edu/fil/Web/

TOR Browser Arrives On Android To Protect Your Secret Searches
https://www.theinquirer.net/inquirer/news/3062424/tor-browser-arrives-on-android-to-protect-your-secret-searches

Tor Coders Harden the Onion Against Surveillance
https://nakedsecurity.sophos.com/2016/06/23/tor-coders-harden-the-onion-against-surveillance/

TOR For Newbies – When Should You Use It?
https://www.makeuseof.com/tag/tor-for-newbies/

TorVPN by Max Eddy
https://uk.pcmag.com/torvpn/19165/review/lockspin-torvpn

Toward the Semantic Deep Web by James Geller, Soon Ae Chun, and Yoo Jung An
https://www.mendeley.com/catalog/toward-semantic-deep-web/

Towards Automatic Incorporation of Search Engines Into A Large-Scale Metasearch Engine
https://www.cs.binghamton.edu/~meng/pub.d/wi2003.pdf

Traffic-Based Feedback on the Web by Jonathan Aizen, Daniel Huttenlocher, Jon Kleinberg, and Antal Novak
https://www.pnas.org/content/101/suppl_1/5254.abstract

Travel Industry and Deep Web: Exclusive Interview with Marcus P. Zillman
https://plrplr.com/90014/deep-web-and-travel-industry-exclusive-interview-with-marcus-p-zillman/

UMBC – AgentNews
https://agents.umbc.edu/

Understanding Metadata: What is Metadata, and What What Is It For? A Primer
https://www.niso.org/publications/understanding-metadata-2017

Understanding the Deep Web In 10 Minutes
https://www.brightplanet.com/2013/03/whitepaper-understanding-the-deep-web-in-10-minutes/

Using the Internet As a Dynamic Resource Tool for Knowledge Discovery 2018
https://www.zillman.us/white-papers/using-the-internet-as-a-dynamic-resource-tool-for-knowledge-discovery/

Web Characterization Activity
https://www.w3.org/WCA/

Web Data Extractors 2019
https://www.WebDataExtractors.com/

Web Pages Search Engine Based on DNS by Wang Liang, Guo Yi-Ping, and Fang Ming
https://arxiv.org/pdf/cs.NI/0403035

WebScales: Towards a Highly Scalable Metasearch Engine
https://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html

What Exactly Is the Mysterious Dark Web and How It Is Different From the Deep Web by Kshitij Pujari
https://tech.firstpost.com/news-analysis/what-exactly-is-the-mysterious-dark-web-and-how-it-is-different-from-deep-web-379668.html

What Happens When You Spend a Week Fishing on the Dark Web? By Anthony Cuthbertson
https://www.independent.co.uk/news/long_reads/dark-web-search-drugs-weapons-crimes-illegal-a8524341.html

What Is The Dark Web?  By David Glance
https://www.iflscience.com/technology/what-dark-web/

What Is the Dark Web and How Does It Work?
https://theconversation.com/what-is-the-dark-web-and-how-does-it-work-63613

What Is the Dark Web and How To Access This Scary Part of Internet by Karandeep Singh
https://www.mobipicker.com/what-is-the-dark-web-how-to-access-the-dark-web/

What is the Dark Web – Our Guide to the Dark Web by Laurie Clarke
https://www.techworld.com/security/what-is-dark-web-3645157/

What Is the Deep Web and How Can You Access It? By Adam Rowe
https://tech.co/what-is-the-deep-web-2018-05

What Is the Deep Web? A WhatIs Podcast 15 Minute Interview with Marcus P. Zillman
http://zillman.blogspot.com/2006/10/what-is-deep-web.html

What Happens When You Spend a Week Fishing On the Dark Web?
https://www.independent.co.uk/news/long_reads/dark-web-search-drugs-weapons-crimes-illegal-a8524341.html

What Is Deep Web? All You Need To Know About It
https://newmexicocourierexpress.com/what-is-deep-web-all-you-need-to-know-about-it/50160

What is the Invisible Web? A Crawler Perspective by Natalia Arroyo, Laboratorio de Internet
http://cybermetrics.wlv.ac.uk/AoIRASIST/arroyo.html

What is the Difference Between the Dark Web and the Deep Web by Laurie Clarke
https://www.techworld.com/data/what-is-difference-between-dark-web-deep-web-3680293/

What It Looks Like On the Dark Side of the Web by Felix Simon
https://medium.com/@FelixSimon/what-it-looks-like-on-the-dark-side-of-the-web-f53ff9f4ef3c

What’s Really On the Dark Web? By Jeff Deal
https://www.wftv.com/news/local/what-s-really-on-the-dark-web-/860574573

Where Your Stolen Personal Details Go Online by Nicholas and Matthew Dunn
https://www.news.com.au/technology/online/security/inside-the-dark-web-where-your-stolen-personal-details-go-online/news-story/1f13c214950187d6412ce4b89bfe42fe

Why the Dark Web Could Be a Growing Threat to Your Small Business by Ben Lobel
https://smallbusiness.co.uk/why-the-dark-web-could-be-a-growing-threat-to-your-small-business-2545071/

Wikipedia – Deep Web
https://en.wikipedia.org/wiki/Deep_web

WISE-Cluster: Clustering E-Commerce Search Engines Automatically by Q. Peng, W. Meng, H. He, C. Yu
https://www.cs.binghamton.edu/~meng/pub.d/PengWIDM04.pdf

CROSS DATABASE ARTICLES

Search Tools Reports: Searching for Text Information in Databases
https://www.searchtools.com/info/database-search.html

UK Web Archiving Consortium
https://www.webarchive.org.uk

CROSS DATABASE SEARCH SERVICES

FDsys – Search Across Multiple Government Databases
https://www.GovInfo.gov/

GovInfo – Discover Government Information
https://www.GovInfo.gov/

King County Library System
https://www.kcls.org/

NLM Gateway Search
https://wwwcf.nlm.nih.gov/hsr_project/home_proj.cfm

SUMSearch 2 [Health Sciences]
https://sumsearch.org/

CROSS DATABASE SEARCH TOOLS

Bright Planet – Deep Web Intelligence
https://brightplanet.com/

Copernic
https://www.copernic.com/

Dieselpoint Java Search and Navigation Software
https://www.dieselpoint.com/

Dublin Core Metadata Initiative (DCMI)
https://www.dublincore.org/

EEVL Xtra – Cross Database Search
https://www.ariadne.ac.uk/issue44/eevl/

Gold Rush – Database Search Tool
https://goldrush.coalliance.org/

MetaLib
https://www.exlibrisgroup.com/category/MetaLibOverview

MetaSearch Initiative
https://www.niso.org/workrooms/mi

MuseGlobal
https://www.museglobal.com/

Peter’s PolySearch Engines
https://www2.hawaii.edu/~jacso/extra/poly-page.html

PBCore – The Public Broadcasting Metadata Dictionary
https://www.pbcore.org/

Search Federal Research and Development
https://www.osti.gov/ 

SRU – Search/Retrieve via URL
https://www.loc.gov/standards/sru

The Flamenco Search Interface Project
https://flamenco.berkeley.edu/

PEER TO PEER (P2P), FILE SHARING, GRID AND MATRIX SEARCH ENGINES

ALPINE Network – SourceForge: Project
https://sourceforge.net/projects/alpine/

Azureus – Vuze Java Bittorrent Client
https://www.vuze.com/

BadBlue [Uncensored News]
https://badblue.com/

Between Rhizomes and Trees: P2P Information Systems by Bryn Loban
https://firstmonday.org/ojs/index.php/fm/article/view/1182

BigChampagne
https://www.bigchampagne.com/

Bitmessage – P2P Communication Protocol To Send Encrypted Messages
https://bitmessage.org/wiki/Main_Page

Bit Torrent Official Site and Search Engine
https://www.BitTorrent.com/

Coral – The Coral P2P Content Distribution Network
https://www.coralcdn.org/

Capn’s PHP Gnutella Search [Only code is available for download]
https://capnbry.net/gnutella/gs.php

Distributed Search Engines
https://en.wikipedia.org/wiki/Distributed_search_engine
https://wiki.p2pfoundation.net/Distributed_Search_Engines

Distributed Search in P2P Networks
https://www.computer.org/csdl/mags/ic/2002/01/w1068-abs.html

FAROO – P2P Web Search
https://www.faroo.com/

FilesOverMiles – Browser to Browser File Sharing (P2P)
https://www.filesovermiles.com/

Filetopia – File sharing tool with public key encryption
https://www.filetopia.org/

Free Haven Project
https://www.freehaven.net

Frost Project – Freenet Messaging and File Sharing Client
https://jtcfrost.sourceforge.net/

FuzzBox: Tangent Research Artificial Intelligence and Robotics
https://tangentresearch.com/news/07252001_p2p_ai.html

GNUnet – Secure P2P Networking – Free Software Foundation (FSF)
https://gnunet.org/

Grid, Distributed and Cloud Computing Resources 2018
https://www.GridResources.info/

GNU GRUB – Multiboot Boot Loader
https://www.gnu.org/software/grub/

Ian Clarke’s Blog
https://blog.locut.us/

Internet Movie Database (IMDb)
https://www.imdb.com/

IPFS Distributed Web
https://ipfs.io/

MoleSter – A Tiny File-Sharing Application
http://ansuz.sooke.bc.ca/software/molester/

MusicBrainZ – Open Music Encyclopedia
https://www.MusicBrainZ.org/

MysterNetworks – The Evolution of Peer-to-Peer
https://www.mysternetworks.com/

OpenNap: Open Source Napster Server
https://opennap.sourceforge.net/

P2P and the Future of Private Copying by Peter K. Yu, Michigan State University College of Law
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=578568

Peer-To-Peer Wikipedia
https://en.wikipedia.org/wiki/Peer-to-peer

Peer to Peer File Sharing – P2P Networking
https://compnetworking.about.com/od/p2ppeertopeer/Peer_to_Peer_File_Sharing_P2P_Networking.htm

Port Knocking
https://www.portknocking.org/

PowerFolder – P2P Whole Folder Synchronization
https://www.powerfolder.com/

Rodi – Tiny P2P Client/Host
https://rodi.sourceforge.net/

Skype
https://www.skype.com/

Slyck – File Sharing News and Info
https://www.slyck.com/

Solid – Reshape the Web
https://solid.inrupt.com/

Stealth Mode Online Privacy Resources 2019
https://www.StealthMode.info/

Swarm – A Transparently Scalable Distributed Programming Language
https://swarmframework.org/

The Anthill Project
https://www.cs.unibo.it/projects/anthill/

The Freenet Project
https://freenetproject.org/

ToPeer
https://www.2peer.com/

Transmission – Fast, Easy and Free BitTorrent Client
https://www.transmissionbt.com

Tribler – Privacy Using Our Tor-Inspired Onion Routing
https://www.tribler.org/

TrustyFiles
https://www.trustyfiles.com/

Understanding BitTorrent: An Experimental Perspective by Arnaud Legout, Guillaume Urvoy-Keller, and Pietro Michiardi
https://hal.inria.fr/inria-00000156/en

WASTE (Secure P2P communication)
https://slackerbitch.free.fr/waste/

YaCy – Distributed P2P Based Web Indexing and Anonymous Search Engine
https://www.yacy.net/

YouServ – A P2P (peer-to-peer) Web Hosting/File Sharing System
https://www.bayardo.org/youserv/

Zebra – Structured Text Indexing and Retrieval
https://www.indexdata.com/zebra

Zilok – Peer To Peer Rental Marketplace
https://zilok.com/

PRESENTATIONS

Deep Web
https://whatis.techtarget.com/definition/deep-Web

Searching the Internet 2018 – The Primer – 2019 Guide to Searching the Internet
https://www.SearchingTheInternet.info/

The Virtual Private Library™ and The Deep Web Video by Melissa Barker
https://zillman.blogspot.com/2009/07/virtual-private-library-and-deep-web.html

RESOURCES – Deep Web Research

6 Dark Web Pricing Trends – It Isn’t Good News
https://craigpeterson.com/radio-show/6-dark-web-pricing-trends-it-isnt-good-news/19603 

Academic and Scholar Search Engines and Sources 2019
https://www.ScholarSearchEngines.com/

A Data-Driven Look At Dark Web Marketplaces
https://www.visualcapitalist.com/data-driven-look-dark-web-marketplaces/

AEON (Automatic Evaluation of ONtologies)
https://code.google.com/p/aeon-project/

AnkaSearch – Meta Search and Deep Web Search Desktop Tool
https://www.ankasoftware.com/ankasearch.html

Anonymous Web Browsing – Wikipedia
https://en.wikipedia.org/wiki/Anonymous_web_browsing

An Up-To-Date Layman’s Guide To Accessing The Deep Web
https://www.fastcompany.com/3026989/an-up-to-date-laymans-guide-to-accessing-the-deep-web

AskReddit – What Are Your Experiences With the Deep Web
https://www.reddit.com/r/AskReddit/comments/lm4dl/reddit_what_are_your_experiences_in_the_deep_web/

Awesome Public Datasets
https://github.com/caesar0301/awesome-public-datasets

BASE – Bielefeld Academic Search Engine
https://www.base-search.net 

Biznar – Deep Federated Search
https://www.deepwebtech.com/2015/08/biznar-a-deep-web-google-alternative/

Bot and Intelligent Agent Research Resources 2019
https://www.BotResearch.info/

BrightPlanet – Deep Web Intelligence
https://www.brightplanet.com/

Catalog of U.S. Government Publications (CGP)
https://catalog.gpo.gov/

Cazoodle – Search, Integrate, and Organize — The Real World
https://www.cazoodle.com/

Censys Search Engine for Devices and Networks
https://censys.io/

CloserLook Search Engine
https://www.closerlooksearch.com/

Creative Commons RDF-Enhanced Search
https://search.creativecommons.org/

Cyber Cemetery
https://govinfo.library.unt.edu/

CyberGhost – One of the World’s Most Trusted and Secure Virtual Private Networks
https://www.cyberghostvpn.com/

Dark Reading
https://www.darkreading.com/

Dark Web Academy – Dark Web Curriculum
https://www.youtube.com/channel/UCJ31aJo8U-ZaRnZ4Y27so_Q

Dark Web Intelligence and Monitoring Service – Perfecta
https://www.prnewswire.com/news-releases/perfecta-expands-into-the-commercial-sector-and-announces-perfecta-lantern-dark-web-intelligence-and-monitoring-service-300648152.html

DarkWeb News – Deep Web
https://darkwebnews.com/category/deep-web/

DarkWeb News – The Ultimate Dark Web Resource
https://darkwebnews.com/

Data Mining Resources 2019
http://www.DataMiningResources.info/

Datasets for Data Mining and Data Science
https://www.kdnuggets.com/datasets/index.html

DeepDive – Analyze Data On a Deeper Level Than Ever Before
https://deepdive.stanford.edu/

Deep.Dot.Web
https://www.DeepDotWeb.com/

Deep Web – Discover Resources That Help You Mine the Deep or Invisible Web Instead of Just Searching the Surface
https://libguides.msubillings.edu/c.php?g=242182&p=1610131

Deep Web Links – Massive Deep Websites Links and URLs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/19104-deep-web-links-massive-deep-websites-links-and-url

Deep Web Research and Discovery Resources 2019
https://www.DeepWebResearch.info/
https://www.DeepWeb.us/

Deep Web Search
https://deep-web.org/

Deep Web Sites
https://www.deepweb-sites.com/

Deep Web Technologies – federated search
https://www.deepwebtech.com/

Deep Web University
https://brightplanet.com/deep-web-university/

Digital Shadows
https://www.digitalshadows.com/

Directory Resources 2018
https://www.DirectoryResources.info/

eGreenBot 2018 – Green Resources Search Engine
https://www.eGreenBot.com/

eHealthcare Bot 2019 Deep Meta Search Engine
https://www.eHealthcareBot.com/

eMarketing Bot 2019 Deep Meta Search Engine
https://www.eMarketingBot.com/

ENDECA
https://www.oracle.com/us/products/applications/commerce/endeca/overview/index.html

Engineering Village
https://www.engineeringvillage.com

EU Open Data Portal
https://data.europa.eu/euodp/en/home

Federated Search Blog
https://federatedsearchblog.com/

Firefox Monitor
https://monitor.firefox.com/

Freely Accessible Databases for the Public
https://www.istl.org/01-winter/internet.html

Google Fusion Tables
https://www.google.com/drive/apps.html#fusiontables

Google Scholar
https://scholar.google.com/

Healthdata.gov
https://www.healthdata.gov/

How To Do a Deep Web Search and Why You Should
https://www.komando.com/tips/396770/how-to-do-a-deep-web-search-and-why-you-should

Indiegogo Datasets
https://webrobots.io/indiegogo-dataset/

Internet Archive
https://www.archive.org/

Invisible Library
https://invislib.blogspot.com/

IOGDS: International Open Government Dataset Search
https://logd.tw.rpi.edu/node/9903

Kapow Web Collector
https://www.automated-info-solutions.com/

Karma – Data Integration Tool
https://www.isi.edu/integration/karma/

KDnuggets: Data Mining, Web Mining, and Knowledge Discovery Guide
https://www.kdnuggets.com/

Kickstarter Datasets
https://webrobots.io/kickstarter-datasets/

Knowledge Discovery 2019
https://www.KnowledgeDiscovery.info/

Knowlesys – Use Deep Data Mining To Find the Intelligence
https://knowlesys.com/

Large-Scale Deep Web Integration: Incomplete Bibliography
https://metaquerier.cs.uiuc.edu/webibib.html

Linked Data – Connect Distributed Data Across the Web
https://linkeddata.org/

LinkingOpenData – W3C SWEO Community Project
https://www.w3.org/wiki/SweoIG/TaskForces/CommunityProjects/LinkingOpenData

List of Academic Databases and Search Engines – Wikipedia
https://en.wikipedia.org/wiki/List_of_academic_databases_and_search_engines

List of Tor Hidden Services from the Wikipedia
https://en.wikipedia.org/wiki/List_of_Tor_hidden_services

MagPortal
https://www.magportal.com/

Mappa.Mundi Magazine
https://mappa.mundi.net/

Mason OER Metafinder
https://mason.deepwebaccess.com/mason__MasonLibrariesOpenEducationResources_5f4/desktop/en/search.html

Mednar – Innovative Medical Search
https://mednar.com/

Mining the Deep Web for Economic Data
https://www.collectiveip.com/grants/NSF:0207603

New Zealand Digital Library
https://www.nzdl.org/

OAI-PMH Implementation Guidelines – Conveying rights expressions about metadata in the OAI-PMH framework
https://www.openarchives.org/OAI/2.0/guidelines-rights.htm

OAIster
https://www.oclc.org/oaister.en.html

OECD.StatExtracts – Complete Databases Available Via OECD’s iLibrary
https://data.oecd.org/

Omnity Insight Engine – Everything Connected
https://www.omnity.io/

OneLook Dictionary Search
https://www.onelook.com/

Onion Browser – An Open-Source Privacy Enhancing Web Browser for iOS
https://mike.tig.as/onionbrowser/

OnionLand Search Engine
https://onionlandsearchengine.com/

Open Archives Initiative
http://www.openarchives.org/

Open Datasets
https://www.DataPortals.org/
https://github.com/caesar0301/awesome-public-datasets
https://www.kaggle.com/datasets
https://www.data.gov/
https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public
https://aws.amazon.com/public-datasets/
https://data.world/
https://data.worldbank.org

Open Educational Resources (OER) Sources 2019 and Search Engine
https://www.OERSources.com/
https://www.OERSearchEngine.com/

OpenIndex – Creating a Public Internet Index
https://www.openindex.org/

Open Source Intelligence
https://www.oss.net/

Open Vulnerability Assessment System (OpenVAS)
https://www.darknet.org.uk/2015/01/openvas-7-released-open-source-vulnerability-scanner/

Privacy Resources 2019
http://www.PrivacyResources.info/

Project Maelstrom – The Internet We Build Next
http://blog.bittorrent.com/category/labs/

re3data.org – 2,000 Data Repositories
https://www.re3data.org/

Recommended Gateway Sites for the Deep Web
https://people.hws.edu/hunter/deepwebgate03.htm

ReportLinker: Industry Reports, Company Profiles and Market Statistics
https://www.reportlinker.com/

SAO/NASA Astrophysics Data System (ADS)
https://adswww.harvard.edu/

reSearcher
https://researcher.sfu.ca/

Science and Technology Sources on the Internet
https://www.loc.gov/rr/scitech/resources.html

Scientific and Technical Information Network (STINET)
https://www.loc.gov/flicc/Exemplars/DTIC/DTIC-STINET.PDF

Science Commons
https://creativecommons.org/science

Science.gov – FirstGov for Science – Government Science Portal
https://www.science.gov/

ScienceResearch.com – Deep Web Search Engine
https://www.scienceresearch.com/

Scientific Data Repository – Real Time Visualization and Exploration Techniques
https://www.mlvis.com/platform.php

SciTech Connect
https://www.osti.gov/scitech/

Scrapinghub Crawls the Deep Web
https://blog.scrapinghub.com/2015/02/24/memex

Scrapy Webcrawler
http://scrapy.org/

SDARTS – A Protocol and Toolkit for Metasearching
https://www.cs.columbia.edu/~gravano/Papers/2001/jcdl01b.pdf

SIMILE Widgets – Free, Open-Source Data Visualization Web Widgets and More
https://simile-widgets.org/

Social Buzz Bot 2019 (PDF download)
http://www.SocialBuzzBot.com/

STN International – Databases in Science and Technology
http://www.stn-international.de/

SurfEasy – Online Privacy
https://www.surfeasy.com/

Swoogle – Semantic Bot
https://en.wikipedia.org/wiki/Swoogle

SWRC Ontology
http://ontoware.org/swrc/

Tails – The Amnesic Incognito Live System
https://tails.boum.org/

TechDeepWeb – How-To Guide to the Deep Web for IT Professionals
http://www.TechDeepWeb.com/

Terbium Labs – Matchlight Proactive Security In an Insecure World
https://terbiumlabs.com/

The 10 Best Deep Web Search Engines to Explore the Hidden Web by Michelle Fuchs
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/17217-top-10-best-deep-web-search-engines-to-explore-hidden-web

The Deep Web: Shutdowns, New Sites, New Tools by Vincenzo Ciancaglini
http://blog.trendmicro.com/trendlabs-security-intelligence/the-deep-web-shutdowns-new-sites-new-tools/

The Enterprise Onion Toolkit
https://github.com/alecmuffett/eotk

The Graph That Knows the World
https://www.datanami.com/2018/10/02/the-graph-that-knows-the-world/

The Invisible Internet Project (I2P)
https://geti2p.net/en/

The World Bank – Data
http://data.worldbank.org/

THOR: Deep Web Data Extraction
http://www.cc.gatech.edu/projects/disl/THOR/

Top 10 Best Deep Web Search Engines To Explore the Hidden Web
https://www.airsassociation.org/services-new/airs-knowledge-network-n/airs-articles/item/18749-top-10-best-deep-web-search-engines-to-explore-hidden-web

Tor Browser Bundle – Anonymity
https://www.torproject.org/projects/torbrowser.html.en

TOR For Newbies – When Should You Use It?
http://www.makeuseof.com/tag/tor-for-newbies/

Tor Project
https://www.torproject.org/

Tor Stack Exchange
http://tor.stackexchange.com/

TRID – The TRIS and ITRD Database (Transportation Research Board)
http://trid.trb.org/

TunnelBear – Simple, Private, Free Access to the Global Internet
https://www.tunnelbear.com/

Twitter/Search #deepweb
https://twitter.com/search?q=%23deepweb

UC Irvine Machine Learning Repository
https://archive.ics.uci.edu/ml/index.php

UNdata – Data Access System To UN Databases
http://data.un.org/

UNESCO Information Services – Databases
http://www.unesco.org/unesdi/index.php/eng/doc/tous.html

United States Census Bureau Research
https://www.census.gov/research/

Useful Tips and Tools to Research the Deep Web
http://www.online-college-blog.com/features/100-useful-tips-and-tools-to-research-the-deep-web/

US Government Web Services and XML Data Sources
http://usgovxml.com/

Virtual Private Networks Directory of Best Services
http://www.makeuseof.com/tag/best-vpn-services/

Web Data Extractors 2019
http://www.WebDataExtractors.com/

WebFountain™ – Analytical engine unstructured data
http://en.wikipedia.org/wiki/IBM_WebFountain

Web IR & IE
https://groups.yahoo.com/neo/groups/webir/info
http://www.webir.org/

WebScales: Towards a Highly Scalable Metasearch Engine
http://www.cs.binghamton.edu/~meng/pub.d/PIreport04.html

What Is a VPN – Beginner’s Guide to Virtual Private Networks
https://www.vpnsrus.com/what-is-a-vpn/

Wolfram Data Respository
https://datarepository.wolframcloud.com/

WTO Statistics Database
http://stat.wto.org/

Zanran – Search the Web for Data and Statistics
http://www.zanran.com/

Zeef – Deep Web Links
https://zeef.com/?query=deep%20web&in=all

Bot and Intelligent Agent Research Resources and Sites

api.ai – Speech Interface for Apps and Devices
http://api.ai/

Applied Soft Computing
http://www.sciencedirect.com/science/journal/15684946

Article Search API – New York Times Articles 1981 to Present
http://developer.nytimes.com/docs/article_search_api

Artificial Intelligence Resources 2019
http://www.AIResources.info/

artoo.js – The Client-Side Scraping Companion
http://medialab.github.io/artoo/

BEN Portal: BioSciEdNet
http://www.biosciednet.org/portal/

BotAnalytics – Analytics for Bots
http://botanalytics.co/

Bot and Intelligent Agent Research Resources 2019
http://www.BotResearch.info/

Botometer
https://botometer.iuni.iu.edu/

Bots, Blogs and News Aggregators 2019
http://www.BotsBlogs.com

Caffe – Deep Learning Framework
http://caffe.berkeleyvision.org/

Chatbase – Analyze and Optimize Bots More Easily
https://chatbase.com/

ChatBottle Search
https://chatbottle.co/

ChatterBot – Convert More Customers Automatically
https://chtb.io/

ChatterBots Resources and Sites 2019
http://www.ChatterBots.info/

Chatfuel – Build a Facebook Bot Without Coding
https://chatfuel.com/

Clara – Digital Employee That Schedules Meetings
https://claralabs.com/

Common Crawl – Open Repository of Web Crawl Data Composed Of Over 5 Billion Freely Available Web Pages
http://www.CommonCrawl.org/

ContusBot – Build Chatbots Super-Charged With Artificial Intelligence
https://www.contus.com/

Contentmine – Text and Data Mining Open Source Tools
http://contentmine.org/

cQuery – Content Query Engine
http://cquery.com/

CrawlTrack – Your Web Statistics Tool
http://www.crawltrack.net/

Create a Crawler – Extract Data From an Entire Website
http://import.io/

Data Mining Resources 2019
http://www.DataMiningResources.info/

Dataminr – Real-time Information Discovery
http://www.dataminr.com/

DataparkSearch Engine – Full-Featured Open Source Web-Based Search Engine
http://www.dataparksearch.org/

DataRobot – Build Better Predictive Models – Faster
http://www.datarobot.com/

Deep Web Research 2019
http://www.deepwebresearch.info/

Design of a Parallel and Distributed Web Search Engine by Salvatore Orlando, Raffaele Perego, and Fabrizio Silvestri
http://arxiv.org/abs/cs.IR/0407053

Dictionary of Algorithms and Data Structures
http://xlinux.nist.gov/dads/

Diffbot Analyze API
https://www.diffbot.com/dev/docs/analyze/

Digital Footprints – Collect Facebook Data
http://digitalfootprints.dk/

Ejenta – Intelligent Agents for Connected Care
https://www.ejenta.com/

Eliza – The Original ChatterBot
http://www-ai.ijs.si/eliza/eliza.html

Engati – Bot Innovation
http://engati.com/

Ethereum Frontier Release – A Decentralized Software Platform
https://www.ethereum.org/

EU Open Data Portal
https://data.europa.eu/euodp/en/home

EventBots by Sciensio
https://www.sciensio.com/eventbots

Facepager – Fetching Public Data From Facebook
https://github.com/strohne/Facepager

FAME (Facilitating Agents in Multiculture Exchange) Project
https://cordis.europa.eu/projects/rcn/58337_en.html

File Information Tool Set (FITS)
http://fitstool.org/

Findo – An Intelligent Search Assistant
https://findo.com/

Foundation for Intelligent Physical Agents
https://www.fipa.org/

Free Robots.txt Generator and Validator
https://www.toolsiseek.com/robots-txt-generator-validator

Friends of ChatBot Coalition
https://www.fcc.ai/

Google Guide
https://www.googleguide.com/

Gupshup – Bot Building Platform
https://www.gupshup.io/

HAIKU – Open Source Operating System That Specificially Targets Personal Computing
https://www.haiku-os.org/

Healthdata.gov
https://www.healthdata.gov/

How to Make a Web Crawler With Selenium
https://www.makeuseof.com/tag/make-web-crawler-selenium/

Huginn – Your Agents Are Standing By
https://github.com/cantino/huginn

HYPHE – Web Corpus Curation Tool Featuring A Research-Driven Web Crawler
http://hyphe.medialab.sciences-po.fr/

IBM Watson Services
https://www.ibm.com/watson/products-services/

idavatars – Intelligent Digital Avitars
http://idavatars.com/

IFR – Federation of Robotics
https://ifr.org/

Imagination Engines
http://www.imagination-engines.com/

Import.io – Turn the Web Into Data With Extractors, Crawlers and Connectors
https://import.io/

Indexing Robot Crawler Checklist
http://www.searchtools.com/robots/robot-checklist.html

Indiegogo Datasets
https://webrobots.io/indiegogo-dataset/

InfoExtractor – Extract Relevant Information from Various Sources Like Blogs, YouTube, and Wikipedia
http://www.infoextractor.org/

Institute for Human and Machine Cognition (IHMC)
http://www.ihmc.us/

Intellexer – Custom Built Search Engines, Knowledge Management Tools, Natural Language Processing
http://www.intellexer.com/

Intelligent Information Systems Research Laboratory
http://iis.ist.psu.edu/

International Journal of Agent-Oriented Software Engineering (IJAOSE)
http://www.inderscience.com/jhome.php?jcode=ijaose

jSEO – Web Crawler For Search Engine Optimization
http://codecanyon.net/item/jseo-web-crawler-for-search-engine-optimization/8770392

Keras: The Python Deep Learning Library
https://keras.io/

Kickstarter Datasets
https://webrobots.io/kickstarter-datasets/

Knowledge Discovery 2019
http://www.knowledgediscovery.info/

LAIR – Laboratory of Applied Informatics Research
http://lair.unc.edu/

List of User-Agents (Spiders, Robots, Crawler, Browser)
http://www.user-agents.org/index.shtml

Ludwig – Find Your Sentence
https://ludwig.guru/

Lurchr – I Keep Track of What’s Shared by Your Team So You Can Stay Focused On Work
https://lurchr.com/

MindMeld – Advanced Artificial Intelligence To Power the New Generation of Intelligent Conversational Interfaces
https://www.mindmeld.com/

Minimal-Intelligence Agents for Bargaining Behaviors in Market-Based Environments by Dave Cliff and Janet Bruten
http://www.hpl.hp.com/techreports/97/HPL-97-91.html

MIT Media Lab: Software Agents
http://agents.media.mit.edu/index.html

Modelling and Mining of Network Information Systems
http://www.mathstat.dal.ca/~mominis/index.html

Motion AI – ChatBots and Artificial Intelligence Made Easy
http://motion.ai/

Mozenda Web Agent Builder – Web Data Extraction
http://www.mozenda.com/

MySpiders
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.3013

NCapture – Capture Web Content
http://www.qsrinternational.com/products_nvivo_add-ons.aspx

Networks and agents Network (NaN)
http://cnets.indiana.edu/groups/nan/

NewsBot – Related News At a Click Of a Button
https://getnewsbot.com/

Open Educational Resources (OER) Sources Search Engine
https://www.OERSearchEngine.com/

oSkope Visual Search
https://oskope.com/

Oxyus Open Source Search Engine
http://sourceforge.net/projects/oxyus/

Postfire – AI Powered Social Media Assistant
https://postfire.co/

ProBot – Automate and Improve Your Workflow
https://probot.github.io/

Rank Dynamics – Make Your Search Dynamic
https://www.rankdynamics.com/

ResearchKit Framework – Medical Research Apps
https://github.com/ResearchKit

Robo Brain – Large Scale Computational System That Learns from Publicly Available Internet Resources
https://robobrain.me/

ReVerb – Open Information Extraction Software
https://reverb.cs.washington.edu/

SoftwareSciensio – AI Specific Events Chatbots
https://www.sciensio.com/eventbots

Scrapple – A Framework For Creating Web Scrapers and Web Crawlers
https://pypi.python.org/pypi/scrapple

Search Engine Robots
https://www.jafsoft.com/searchengines/webbots.html

Search Engine Watch News
https://www.searchenginewatch.com/

Search Tools – Information Guides and News
https://www.searchtools.com/

SeerSuite – CiteSeerX Toolkit
http://sourceforge.net/projects/citeseerx/

Semantic Web
http://www.semanticweb.org/

ShoppingBots 2019
http://www.ShoppingBots.info/

Siri – Your Virtual Personal Assistant
https://www.apple.com/ios/siri/

Sisense Bot Framework
https://www.sisense.com/product/impact/bot/

Smarter Bots 2019
http://www.SmarterBots.com/

SocialBuzzBot 2019- The Business and Social Intelligence Search Engine for Information Discovery from Social Communities
https://www.SocialBuzzBot.com/

SocSciBot – Social Sciences Link Analysis Research
http://socscibot.wlv.ac.uk/

Spidering Hacks
https://www.oreilly.com/catalog/spiderhks/

STACKS – Social Media Tracker, Analyzer, & Collector Toolkit at Syracuse
https://github.com/bitslabsyr/stack

Structure and Interpretation of Computer Programs – Video Lectures by HalAbelson and Gerald Jay Sussman
https://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/

SupportBots – Chatbots for MSP and IT Service Providers
https://www.supportbots.io/

Supybot, A Superb Python IRC Bot
https://freecode.com/projects/supybot?branch_id=31808&release_id=181322

Surf Canyon Search Engine
https://www.surfcanyon.com/

Swirlds – The Platform for Distributed Applications
https://www.swirlds.com/

Swoogle – Semantic Bot
https://en.wikipedia.org/wiki/Swoogle

TextRunner Search – Searches Hundreds of Millions of Assertions Extracted from 500 Million High-Quality Web Pages
https://openie.cs.washington.edu/

The Essential Landscape of Enterprise AI Companies
https://www.topbots.com/essential-landscape-overview-enterprise-artificial-intelligence/

The Intelligent Software Agents Lab
https://www.cs.cmu.edu/~softagents/

The Lemur Toolkit – Language Modeling and Information Retrieval Research
https://www.lemurproject.org/

The Search Engine Project (TSEP)
https://freecode.com/projects/tsep

The Simon Lavern Page
https://www.simonlaven.com/

Transcriptic – Discovery Biology On Demand
https://www.transcriptic.com/

UC Irvine Machine Learning Repository
https://archive.ics.uci.edu/ml/index.php

UMBC AgentWeb
https://agents.umbc.edu/

UMBC eBiquity
https://ebiquity.umbc.edu/

Universe – Measurement and Training for Artificial Intelligence
https://universe.openai.com/

Web Curator Tool (WCT)
https://webcurator.sourceforge.net/

Web Data Extractors 2019
https://www.WebDataExtractors.com/

Web Intelligence Consortium
https://wi-consortium.org/

Web IR & IE
https://groups.yahoo.com/neo/groups/webir/info
https://www.webir.org/

WolframAlpha Computational Knowledge Engine – Trillions of Pieces of Curated Data and Millions of Lines of Algorithms
https://www.wolframalpha.com/

Xenioo – Professional Chatbot Platform for Your Business
https://www.xenioo.com/

Zanran – Search the Web for Data and Statistics
https://www.zanran.com/

Posted in: Big Data, Business Research, Competitive Intelligence, Discovery, Encryption, Privacy, Search Engines, Search Strategies