Wikipedia trail¶

  • Look at the page Wikipedia:Getting_to_Philosophy on the right you can see a script doing the crawl.
  • Do a Google search for the script scraper.py

You should get to this page. Now copy the script and try to get it to work!!!

Discussion¶

Web pages are created from databases by robots. Sometimes the format of the web page changes and so the crawler won't find what you are looking for. This script no longer works but it should be easy for you to fix it.

What-back-end-infrastructure for wikipedia

Probably Wikimedia servers is a good place to start looking at how the backend works. Its pretty complex. Some of the components include

  • Master SQL database running MariaDB.
  • Several Slave databases also running MariaDB.
  • Application Servers running Apache and MediaWiki.
  • HTTP Cache servers which serve pre compiled pages. Running Varnish HTTP Cache
  • Load balancing systems. running LVS
In [13]:
import time, re
import urllib

#import bs4
import requests

start_url = "https://en.wikipedia.org/wiki/Special:Random"
target_url = "https://en.wikipedia.org/wiki/Philosophy"

# I changed so I don't use bs4
# extract paragraphs
pp = re.compile('<p>(.*?)</p>', re.DOTALL)
pax = re.compile(r'<a href="(.*?)".*?>(.*?)</a>', re.DOTALL)
# this is a better pattern extract links outside of the page
pax = re.compile(r'<a href="([^#]*?)".*?>(.*?)</a>', re.DOTALL)

def find_first(url):
    # this function defines 2 functions
    # in its **local** scope 
    
    def bad_lk(x):
        return ':' in x
        return ('Help:' in x) or ('File:' in x)
    
    def remove_text_between_parens(text):
        #https://stackoverflow.com/questions/37528373/
        n = 1  
        while n:
            # remove non-nested/flat balanced parts
            text, n = re.subn(r'\([^()]*\)', '', text) 
        return text
    
    response = requests.get(url)

    html = remove_text_between_parens(response.text)
    
    # sometimes there are none like for Categories
    paras = pp.findall(html)
    if not paras: return
    
    # this is a primitive filter 
    # I added this  so that we never get to Categories
    lks = [lk for lk in pax.findall(''.join(paras)) if not bad_lk(lk[0])]
    if not lks: return
    
    article_link = lks[0]
    print(article_link)
    
    #I wouldn't import urllib just to do this
    return urllib.parse.urljoin('https://en.wikipedia.org/', article_link[0])

def continue_crawl(search_history, target_url, max_steps=25):
    if search_history[-1] == target_url:
        print("We've found the target article!")
        return False
    elif len(search_history) > max_steps:
        print("The search has gone on suspiciously long, aborting search!")
        return False
    elif search_history[-1] in search_history[:-1]:
        print("We've arrived at an article we've already seen, aborting search!")
        return False
    else:
        return True

article_chain = [start_url]

while continue_crawl(article_chain, target_url): 
    
    first_link = find_first(article_chain[-1])
    if not first_link:
        print("We've arrived at an article with no links, aborting search!")
        break

    article_chain.append(first_link)
    # slow down otherwise wiki server will block you
    time.sleep(2) 

('/wiki/Altair-class_destroyer_tender', '<i>Altair</i>&#x20;class')
('/wiki/United_States_Navy', 'United States Navy')
('/wiki/Naval_warfare', 'maritime')
('/wiki/Combat', 'combat')
('/wiki/Violence', 'violent')
('/wiki/World_Health_Organization', 'World Health Organization')
('/wiki/Geographic_coordinate_system', 'Coordinates')
('/wiki/Spherical_coordinate_system', 'spherical')
('/wiki/Mathematics', 'mathematics')
('/wiki/Number_theory', 'number theory')
('/wiki/Pure_mathematics', 'pure mathematics')
('/wiki/Mathematics', 'mathematics')
We've arrived at an article we've already seen, aborting search!
In [29]:
url = "https://en.wikipedia.org/wiki/Epistemology"

pax = re.compile(r'<a href="([^#]*?)".*?>(.*?)</a>', re.DOTALL)

response = requests.get(url)
html = response.text

html = re.sub('\(.*\)', '', response.text)


pax.findall(' '.join([ x for x in pp.findall(html) ]) )
#if 'wiki' in x[0]]
Out[29]:
[('/wiki/Religious_philosophy', 'Traditions by religion'),
 ('/wiki/Outline_of_philosophy', 'branch of philosophy'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Ethics', 'ethics'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Metaphysics', 'metaphysics'),
 ('/wiki/Justification_', 'epistemic justification'),
 ('/wiki/Reason', 'rationality'),
 ('/wiki/Belief', 'belief'),
 ('/wiki/Episteme', '<i>epistēmē</i>'),
 ('/wiki/Discourse', 'discourse'),
 ('/wiki/Johann_Fichte', 'Johann Fichte'),
 ('/wiki/Bernard_Bolzano', 'Bernard Bolzano'),
 ('/wiki/Jean_Paul', 'Jean Paul'),
 ('/wiki/James_Frederick_Ferrier', 'James Frederick Ferrier'),
 ('/wiki/Ontology', 'ontology'),
 ('/wiki/Philosophy_of_science', 'philosophy of science'),
 ('/wiki/%C3%89mile_Meyerson', 'Émile Meyerson'),
 ('/wiki/John_Locke', 'John Locke'),
 ('/wiki/An_Essay_Concerning_Human_Understanding',
  'Essay Concerning Human Understanding'),
 ('/wiki/Ancient_Greek_philosophy', 'Ancient Greek philosophers'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Republic_', 'Theaetetus'),
 ('/wiki/Meno', 'Meno'),
 ('/wiki/Aristotle', 'Aristotle'),
 ('/wiki/Hellenistic_philosophy', 'Hellenistic period'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Pyrrhonism', 'Pyrrhonian skepticism'),
 ('/wiki/Pyrrho', 'Pyrrho'),
 ('/wiki/Sextus_Empiricus', 'Sextus Empiricus'),
 ('/wiki/Eudaimonia', 'eudaimonia'),
 ('/wiki/Dogma', 'dogmas'),
 ('/wiki/Stoicism', 'Stoicism'),
 ('/wiki/Epicureanism', 'Epicureanism'),
 ('/wiki/Academic_skepticism', 'Academic skepticism'),
 ('/wiki/Carneades', 'Carneades'),
 ('/wiki/Arcesilaus', 'Arcesilaus'),
 ('/wiki/Platonic_Academy', 'Platonic Academy'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Hindu_philosophy', 'ancient Indian philosophy'),
 ('/wiki/%C5%9Arama%E1%B9%87a', 'Śramaṇa movement'),
 ('/wiki/Pre-sectarian_Buddhism', 'early Buddhism'),
 ('/wiki/Jainism', 'Jainism'),
 ('/wiki/%C4%80j%C4%ABvika', 'Ājīvika'),
 ('/wiki/Medieval_philosophy', 'Medieval philosophers'),
 ('/wiki/Thomas_Aquinas', 'Thomas Aquinas'),
 ('/wiki/John_Duns_Scotus', 'John Duns Scotus'),
 ('/wiki/William_of_Ockham', 'William of Ockham'),
 ('/wiki/Islamic_Golden_Age', 'Islamic Golden Age'),
 ('/wiki/Al-Ghazali', 'Al-Ghazali'),
 ('/wiki/Modern_philosophy', 'early modern period'),
 ('/wiki/Empiricism', 'empiricists'),
 ('/wiki/Transcendental_idealism', 'transcendental idealism'),
 ('/wiki/Barry_Stroud', 'Barry Stroud'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Bertrand_Russell', 'Bertrand Russell'),
 ('/wiki/Descriptive_knowledge', 'knowledge by description'),
 ('/wiki/Gilbert_Ryle', 'Gilbert Ryle'),
 ('/wiki/The_Concept_of_Mind', 'The Concept of Mind'),
 ('/wiki/Michael_Polanyi', 'Michael Polanyi'),
 ('/wiki/Infinite_regress', 'infinite regress'),
 ('/wiki/Organon', 'Organon'),
 ('/wiki/Rationalist', 'rationalist'),
 ('/wiki/Empiricist', 'empiricist'),
 ('/wiki/Wikipedia:Citation_needed',
  '<span title="This claim needs references to reliable sources. ">citation needed</span>'),
 ('/wiki/Proposition', 'proposition'),
 ('/wiki/Truth', 'Truth'),
 ('/wiki/Correspondence_theory_of_truth', 'correspondence theory of truth'),
 ('/wiki/Truth-bearer', 'truth-bearer'),
 ('/wiki/Propositions', 'propositions'),
 ('/wiki/Sentence_', 'clarification needed</span>'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Principle_of_bivalence', 'bivalent'),
 ('/wiki/Criteria_of_truth', 'criteria of truth'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Absolute_', 'absolute'),
 ('/wiki/Knowledge_relativity', 'relative'),
 ('/wiki/Perception', 'perceptual experience'),
 ('/wiki/Reason', 'reason'),
 ('/wiki/Testimony', 'testimony'),
 ('/wiki/Alvin_Goldman', 'Alvin Goldman'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Natural_language', 'natural language'),
 ('/wiki/Awareness', 'awareness'),
 ('/wiki/Fact', 'facts'),
 ('/wiki/Mars', 'Mars'),
 ('/wiki/Skill', 'skills'),
 ('/wiki/Experience', 'experiential'),
 ('/wiki/Daniel_Craig', 'Daniel Craig'),
 ('/wiki/Descriptive_knowledge', 'descriptive knowledge'),
 ('/wiki/Proposition', 'proposition'),
 ('/wiki/Reality', 'reality'),
 ('/wiki/Analysis_of_knowledge', 'analysis of knowledge'),
 ('/wiki/Essence', 'essential'),
 ('/wiki/Philosophical_methodology', 'methodology'),
 ('/wiki/Ordinary_language_philosophy', 'ordinary language'),
 ('/wiki/Ancient_Greek_philosophy', 'ancient Greek philosophy'),
 ('/wiki/Mental_state', 'mental state'),
 ('/wiki/Superstition', 'superstition'),
 ('/wiki/Luck', 'lucky'),
 ('/wiki/Fallacy', 'faulty reasoning'),
 ('/wiki/Memory', 'memory'),
 ('/wiki/Edmund_Gettier', 'Edmund Gettier'),
 ('/wiki/Potemkin_village', 'barn façades'),
 ('/wiki/Defeater', 'defeater'),
 ('/wiki/Virtue_epistemology', 'Virtue theorists'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Meno', 'Meno'),
 ('/wiki/Virtue_epistemology', 'virtue epistemology'),
 ('/wiki/Ernest_Sosa', 'Ernest Sosa'),
 ('/wiki/John_Greco_, rather than merely the properties of propositions and propositional mental attitudes.\n The value problem has been presented as an argument against epistemic <a href=',
  'reliabilism'),
 ('/wiki/Linda_Zagzebski', 'Linda Zagzebski'),
 ('/wiki/Richard_Swinburne', 'Richard Swinburne'),
 ('/wiki/Jonathan_Kvanvig', 'Kvanvig'),
 ('/wiki/Perception', 'perception'),
 ('/wiki/Reason', 'reason'),
 ('/wiki/Memory', 'memory'),
 ('/wiki/A_priori_and_a_posteriori', 'a priori'),
 ('/wiki/Rationalism', 'rationalism'),
 ('/wiki/Intuition', 'intuition'),
 ('/wiki/Innatism', 'innatism'),
 ('/wiki/Empiricism', 'empiricism'),
 ('/wiki/Wikipedia:Please_clarify',
  '<span title="The text near this tag may need clarification or removal of jargon. ">clarification needed</span>'),
 ('/wiki/Evolutionary_psychology', 'Evolutionary psychology'),
 ('/wiki/Tabula_rasa', 'tabula rasa'),
 ('/wiki/Immanuel_Kant', 'Immanuel Kant'),
 ('/wiki/Critique_of_Pure_Reason', 'Critique of Pure Reason'),
 ('/wiki/Logical_truth', 'necessarily true'),
 ('/wiki/Meaning_, then there can only be two kinds of successful inquiry: Logico-mathematical inquiry, which investigates what is true by definition, and empirical inquiry, which investigates what is true in the world. Most notably, this would exclude the possibility that branches of philosophy like <a href=',
  'metaphysics'),
 ('/wiki/Willard_Van_Orman_Quine', 'Willard Van Orman Quine'),
 ('/wiki/Two_Dogmas_of_Empiricism', 'Two Dogmas of Empiricism'),
 ('/wiki/Philosophy_of_science', 'philosophy of science'),
 ('/wiki/Regress_argument', 'regress problem'),
 ('/wiki/Skepticism', 'skepticism'),
 ('/wiki/I_think,_therefore_I_am', 'I think, therefore I am'),
 ('/wiki/Foundationalism', 'Foundationalists'),
 ('/wiki/Coherentism', 'coherentism'),
 ('/wiki/Coherentism', 'coherentists'),
 ('/wiki/Correspondence_theory_of_truth', 'corresponds'),
 ('/wiki/Circular_argument', '[c]ircular arguments'),
 ('/wiki/Infinitism', 'infinitism'),
 ('/wiki/Peter_D._Klein', 'Peter Klein'),
 ('/wiki/Foundherentism', 'foundherentism'),
 ('/wiki/Susan_Haack', 'Susan Haack'),
 ('/wiki/Empiricism', 'Empiricism'),
 ('/wiki/Perception', 'perceptual observations'),
 ('/wiki/Sense', 'senses'),
 ('/wiki/Mathematics', 'mathematics'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/British_empiricism', 'British empiricism'),
 ('/wiki/Phenomenalism', 'phenomenalism'),
 ('/wiki/Scottish_common_sense_realism', 'common sense philosophy'),
 ('/wiki/Sense_data', 'sense data'),
 ('/wiki/John_Locke', 'John Locke'),
 ('/wiki/David_Hume', 'David Hume'),
 ('/wiki/George_Berkeley', 'George Berkeley'),
 ('/wiki/Francis_Bacon', 'Francis Bacon'),
 ('/wiki/John_Stuart_Mill', 'John Stuart Mill'),
 ('/wiki/Rudolf_Carnap', 'Rudolf Carnap'),
 ('/wiki/Bertrand_Russell', 'Bertrand Russell'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Mathematics', 'mathematics'),
 ('/wiki/Ethics', 'ethics'),
 ('/wiki/Metaphysics', 'metaphysics'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Baruch_Spinoza', 'Baruch Spinoza'),
 ('/wiki/Gottfried_Leibniz', 'Gottfried Leibniz'),
 ('/wiki/Hellenistic_philosophy', 'Hellenistic period in philosophy'),
 ('/wiki/Pyrrhonism', 'Pyrrhonism'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Madhyamika', 'Madhyamika'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Certainty', 'certainty'),
 ('/wiki/Certainty', 'certainty'),
 ('/wiki/Fallibilism', 'fallible'),
 ('/wiki/Acatalepsy', 'impossible'),
 ('/wiki/Agrippa%27s_trilemma', "Agrippa's trilemma"),
 ('/wiki/Pyrrho_of_Elis', 'Pyrrho of Elis'),
 ('/wiki/Sextus_Empiricus', 'Sextus Empiricus'),
 ('/wiki/Evil_demon', 'Cartesian evil demon problem'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Ontological_argument', 'ontological argument'),
 ('/wiki/Direct_realism', 'direct realism'),
 ('/wiki/Disjunctivism', 'disjunctivism'),
 ('/wiki/Scottish_common_sense_realism', 'common sense philosophy'),
 ('/wiki/Pragmatism', 'pragmatism'),
 ('/wiki/Fideism', 'fideism'),
 ('/wiki/Fictionalism', 'fictionalism'),
 ('/wiki/Pragmatism', 'Pragmatism'),
 ('/wiki/Charles_Sanders_Peirce', 'Charles Sanders Peirce'),
 ('/wiki/William_James', 'William James'),
 ('/wiki/John_Dewey', 'John Dewey'),
 ('/wiki/Richard_Rorty', 'Richard Rorty'),
 ('/wiki/Hilary_Putnam', 'Hilary Putnam'),
 ('/wiki/Neopragmatism', 'neopragmatists'),
 ('/wiki/Nicholas_Rescher', 'Nicholas Rescher'),
 ('/wiki/Robert_Brandom', 'Robert Brandom'),
 ('/wiki/Susan_Haack', 'Susan Haack'),
 ('/wiki/Cornel_West', 'Cornel West'),
 ('/wiki/Naturalized_epistemology', 'naturalized epistemology'),
 ('/wiki/W.V.O._Quine', 'W.V.O. Quine'),
 ('/wiki/Hilary_Kornblith', 'Hilary Kornblith'),
 ('/wiki/Contextualism', 'contextualism'),
 ('/wiki/Ontological', 'ontological'),
 ('/wiki/Social_constructivism', 'Constructivism'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Truth', 'truth'),
 ('/wiki/Idealism', 'Idealism'),
 ('/wiki/Ontological', 'ontological'),
 ('/wiki/Metaphysical', 'metaphysical'),
 ('/wiki/Transcendental_idealism', 'transcendental idealism'),
 ('/wiki/Bayesian_epistemology', 'Bayesian epistemology'),
 ('/wiki/Thomas_Bayes', 'Thomas Bayes'),
 ('/wiki/Subjective_probability', 'subjective probabilities'),
 ('/wiki/Probability_theory', 'probability theory'),
 ('/wiki/Rationality', 'rationality'),
 ('/wiki/Dutch_book', 'Dutch books'),
 ('/wiki/Feminist_epistemology', 'Feminist epistemology'),
 ('/wiki/Feminist_theory', 'feminist theory'),
 ('/wiki/Miranda_Fricker', 'Miranda Fricker'),
 ('/wiki/Sandra_Harding', 'Sandra Harding'),
 ('/wiki/Elizabeth_S._Anderson', 'Elizabeth Anderson'),
 ('/wiki/Social_epistemology', 'social epistemology'),
 ('/wiki/Boaventura_de_Sousa_Santos', 'Boaventura de Sousa Santos'),
 ('/wiki/Indian_philosophy', 'Indian schools of philosophy'),
 ('/wiki/Hindu_philosophy', 'Hindu'),
 ('/wiki/Nyaya', 'Nyaya'),
 ('/wiki/Carvaka', 'Carvaka'),
 ('/wiki/Jain_philosophy', 'Jain'),
 ('/wiki/Buddhist_philosophy', 'Buddhist'),
 ('/wiki/Carvaka', 'Carvaka'),
 ('/wiki/Materialists', 'materialists'),
 ('/wiki/Empiricists', 'empiricists'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Dharmakirti', 'Dharmakirti'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Perspectivism', 'perspectivism'),
 ('/wiki/Jain_epistemology', 'Jain epistemology'),
 ('/wiki/Testimony', 'testimony'),
 ('/wiki/Decision_theory', 'decision theory'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Probability_theory', 'probability theory'),
 ('/wiki/Computability_theory', 'computability theory'),
 ('/wiki/Philosophy', 'philosophy'),
 ('/wiki/Computer_science', 'computer science'),
 ('/wiki/Economics', 'economics'),
 ('/wiki/Statistics', 'statistics'),
 ('/wiki/Metaphilosophical', 'metaphilosophical'),
 ('/wiki/Philosophical_method', 'methods')]
In [15]:
txt = ''.join(pp.findall(html))
pax.findall(txt)
Out[15]:
[('/wiki/Religious_philosophy', 'Traditions by religion'),
 ('/wiki/Outline_of_philosophy', 'branch of philosophy'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Ethics', 'ethics'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Metaphysics', 'metaphysics'),
 ('/wiki/Justification_', 'epistemic justification'),
 ('/wiki/Reason', 'rationality'),
 ('/wiki/Belief', 'belief'),
 ('/wiki/Episteme', '<i>epistēmē</i>'),
 ('/wiki/Discourse', 'discourse'),
 ('/wiki/Johann_Fichte', 'Johann Fichte'),
 ('/wiki/Bernard_Bolzano', 'Bernard Bolzano'),
 ('/wiki/Jean_Paul', 'Jean Paul'),
 ('/wiki/James_Frederick_Ferrier', 'James Frederick Ferrier'),
 ('/wiki/Ontology', 'ontology'),
 ('/wiki/Philosophy_of_science', 'philosophy of science'),
 ('/wiki/%C3%89mile_Meyerson', 'Émile Meyerson'),
 ('/wiki/John_Locke', 'John Locke'),
 ('/wiki/An_Essay_Concerning_Human_Understanding',
  'Essay Concerning Human Understanding'),
 ('/wiki/Ancient_Greek_philosophy', 'Ancient Greek philosophers'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Republic_', 'Theaetetus'),
 ('/wiki/Meno', 'Meno'),
 ('/wiki/Aristotle', 'Aristotle'),
 ('/wiki/Hellenistic_philosophy', 'Hellenistic period'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Pyrrhonism', 'Pyrrhonian skepticism'),
 ('/wiki/Pyrrho', 'Pyrrho'),
 ('/wiki/Sextus_Empiricus', 'Sextus Empiricus'),
 ('/wiki/Eudaimonia', 'eudaimonia'),
 ('/wiki/Dogma', 'dogmas'),
 ('/wiki/Stoicism', 'Stoicism'),
 ('/wiki/Epicureanism', 'Epicureanism'),
 ('/wiki/Academic_skepticism', 'Academic skepticism'),
 ('/wiki/Carneades', 'Carneades'),
 ('/wiki/Arcesilaus', 'Arcesilaus'),
 ('/wiki/Platonic_Academy', 'Platonic Academy'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Hindu_philosophy', 'ancient Indian philosophy'),
 ('/wiki/%C5%9Arama%E1%B9%87a', 'Śramaṇa movement'),
 ('/wiki/Pre-sectarian_Buddhism', 'early Buddhism'),
 ('/wiki/Jainism', 'Jainism'),
 ('/wiki/%C4%80j%C4%ABvika', 'Ājīvika'),
 ('/wiki/Medieval_philosophy', 'Medieval philosophers'),
 ('/wiki/Thomas_Aquinas', 'Thomas Aquinas'),
 ('/wiki/John_Duns_Scotus', 'John Duns Scotus'),
 ('/wiki/William_of_Ockham', 'William of Ockham'),
 ('/wiki/Islamic_Golden_Age', 'Islamic Golden Age'),
 ('/wiki/Al-Ghazali', 'Al-Ghazali'),
 ('/wiki/Modern_philosophy', 'early modern period'),
 ('/wiki/Empiricism', 'empiricists'),
 ('/wiki/Transcendental_idealism', 'transcendental idealism'),
 ('/wiki/Barry_Stroud', 'Barry Stroud'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Bertrand_Russell', 'Bertrand Russell'),
 ('/wiki/Descriptive_knowledge', 'knowledge by description'),
 ('/wiki/Gilbert_Ryle', 'Gilbert Ryle'),
 ('/wiki/The_Concept_of_Mind', 'The Concept of Mind'),
 ('/wiki/Michael_Polanyi', 'Michael Polanyi'),
 ('/wiki/Infinite_regress', 'infinite regress'),
 ('/wiki/Organon', 'Organon'),
 ('/wiki/Rationalist', 'rationalist'),
 ('/wiki/Empiricist', 'empiricist'),
 ('/wiki/Wikipedia:Citation_needed',
  '<span title="This claim needs references to reliable sources. ">citation needed</span>'),
 ('/wiki/Proposition', 'proposition'),
 ('/wiki/Truth', 'Truth'),
 ('/wiki/Correspondence_theory_of_truth', 'correspondence theory of truth'),
 ('/wiki/Truth-bearer', 'truth-bearer'),
 ('/wiki/Propositions', 'propositions'),
 ('/wiki/Sentence_', 'clarification needed</span>'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Principle_of_bivalence', 'bivalent'),
 ('/wiki/Criteria_of_truth', 'criteria of truth'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Absolute_', 'absolute'),
 ('/wiki/Knowledge_relativity', 'relative'),
 ('/wiki/Perception', 'perceptual experience'),
 ('/wiki/Reason', 'reason'),
 ('/wiki/Testimony', 'testimony'),
 ('/wiki/Alvin_Goldman', 'Alvin Goldman'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Natural_language', 'natural language'),
 ('/wiki/Awareness', 'awareness'),
 ('/wiki/Fact', 'facts'),
 ('/wiki/Mars', 'Mars'),
 ('/wiki/Skill', 'skills'),
 ('/wiki/Experience', 'experiential'),
 ('/wiki/Daniel_Craig', 'Daniel Craig'),
 ('/wiki/Descriptive_knowledge', 'descriptive knowledge'),
 ('/wiki/Proposition', 'proposition'),
 ('/wiki/Reality', 'reality'),
 ('/wiki/Analysis_of_knowledge', 'analysis of knowledge'),
 ('/wiki/Essence', 'essential'),
 ('/wiki/Philosophical_methodology', 'methodology'),
 ('/wiki/Ordinary_language_philosophy', 'ordinary language'),
 ('/wiki/Ancient_Greek_philosophy', 'ancient Greek philosophy'),
 ('/wiki/Mental_state', 'mental state'),
 ('/wiki/Superstition', 'superstition'),
 ('/wiki/Luck', 'lucky'),
 ('/wiki/Fallacy', 'faulty reasoning'),
 ('/wiki/Memory', 'memory'),
 ('/wiki/Edmund_Gettier', 'Edmund Gettier'),
 ('/wiki/Potemkin_village', 'barn façades'),
 ('/wiki/Defeater', 'defeater'),
 ('/wiki/Virtue_epistemology', 'Virtue theorists'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Meno', 'Meno'),
 ('/wiki/Virtue_epistemology', 'virtue epistemology'),
 ('/wiki/Ernest_Sosa', 'Ernest Sosa'),
 ('/wiki/John_Greco_, rather than merely the properties of propositions and propositional mental attitudes.\nThe value problem has been presented as an argument against epistemic <a href=',
  'reliabilism'),
 ('/wiki/Linda_Zagzebski', 'Linda Zagzebski'),
 ('/wiki/Richard_Swinburne', 'Richard Swinburne'),
 ('/wiki/Jonathan_Kvanvig', 'Kvanvig'),
 ('/wiki/Perception', 'perception'),
 ('/wiki/Reason', 'reason'),
 ('/wiki/Memory', 'memory'),
 ('/wiki/A_priori_and_a_posteriori', 'a priori'),
 ('/wiki/Rationalism', 'rationalism'),
 ('/wiki/Intuition', 'intuition'),
 ('/wiki/Innatism', 'innatism'),
 ('/wiki/Empiricism', 'empiricism'),
 ('/wiki/Wikipedia:Please_clarify',
  '<span title="The text near this tag may need clarification or removal of jargon. ">clarification needed</span>'),
 ('/wiki/Evolutionary_psychology', 'Evolutionary psychology'),
 ('/wiki/Tabula_rasa', 'tabula rasa'),
 ('/wiki/Immanuel_Kant', 'Immanuel Kant'),
 ('/wiki/Critique_of_Pure_Reason', 'Critique of Pure Reason'),
 ('/wiki/Logical_truth', 'necessarily true'),
 ('/wiki/Meaning_, then there can only be two kinds of successful inquiry: Logico-mathematical inquiry, which investigates what is true by definition, and empirical inquiry, which investigates what is true in the world. Most notably, this would exclude the possibility that branches of philosophy like <a href=',
  'metaphysics'),
 ('/wiki/Willard_Van_Orman_Quine', 'Willard Van Orman Quine'),
 ('/wiki/Two_Dogmas_of_Empiricism', 'Two Dogmas of Empiricism'),
 ('/wiki/Philosophy_of_science', 'philosophy of science'),
 ('/wiki/Regress_argument', 'regress problem'),
 ('/wiki/Skepticism', 'skepticism'),
 ('/wiki/I_think,_therefore_I_am', 'I think, therefore I am'),
 ('/wiki/Foundationalism', 'Foundationalists'),
 ('/wiki/Coherentism', 'coherentism'),
 ('/wiki/Coherentism', 'coherentists'),
 ('/wiki/Correspondence_theory_of_truth', 'corresponds'),
 ('/wiki/Circular_argument', '[c]ircular arguments'),
 ('/wiki/Infinitism', 'infinitism'),
 ('/wiki/Peter_D._Klein', 'Peter Klein'),
 ('/wiki/Foundherentism', 'foundherentism'),
 ('/wiki/Susan_Haack', 'Susan Haack'),
 ('/wiki/Empiricism', 'Empiricism'),
 ('/wiki/Perception', 'perceptual observations'),
 ('/wiki/Sense', 'senses'),
 ('/wiki/Mathematics', 'mathematics'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/British_empiricism', 'British empiricism'),
 ('/wiki/Phenomenalism', 'phenomenalism'),
 ('/wiki/Scottish_common_sense_realism', 'common sense philosophy'),
 ('/wiki/Sense_data', 'sense data'),
 ('/wiki/John_Locke', 'John Locke'),
 ('/wiki/David_Hume', 'David Hume'),
 ('/wiki/George_Berkeley', 'George Berkeley'),
 ('/wiki/Francis_Bacon', 'Francis Bacon'),
 ('/wiki/John_Stuart_Mill', 'John Stuart Mill'),
 ('/wiki/Rudolf_Carnap', 'Rudolf Carnap'),
 ('/wiki/Bertrand_Russell', 'Bertrand Russell'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Mathematics', 'mathematics'),
 ('/wiki/Ethics', 'ethics'),
 ('/wiki/Metaphysics', 'metaphysics'),
 ('/wiki/Plato', 'Plato'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Baruch_Spinoza', 'Baruch Spinoza'),
 ('/wiki/Gottfried_Leibniz', 'Gottfried Leibniz'),
 ('/wiki/Hellenistic_philosophy', 'Hellenistic period in philosophy'),
 ('/wiki/Pyrrhonism', 'Pyrrhonism'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Madhyamika', 'Madhyamika'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Certainty', 'certainty'),
 ('/wiki/Certainty', 'certainty'),
 ('/wiki/Fallibilism', 'fallible'),
 ('/wiki/Acatalepsy', 'impossible'),
 ('/wiki/Agrippa%27s_trilemma', "Agrippa's trilemma"),
 ('/wiki/Pyrrho_of_Elis', 'Pyrrho of Elis'),
 ('/wiki/Sextus_Empiricus', 'Sextus Empiricus'),
 ('/wiki/Evil_demon', 'Cartesian evil demon problem'),
 ('/wiki/Ren%C3%A9_Descartes', 'René Descartes'),
 ('/wiki/Ontological_argument', 'ontological argument'),
 ('/wiki/Direct_realism', 'direct realism'),
 ('/wiki/Disjunctivism', 'disjunctivism'),
 ('/wiki/Scottish_common_sense_realism', 'common sense philosophy'),
 ('/wiki/Pragmatism', 'pragmatism'),
 ('/wiki/Fideism', 'fideism'),
 ('/wiki/Fictionalism', 'fictionalism'),
 ('/wiki/Pragmatism', 'Pragmatism'),
 ('/wiki/Charles_Sanders_Peirce', 'Charles Sanders Peirce'),
 ('/wiki/William_James', 'William James'),
 ('/wiki/John_Dewey', 'John Dewey'),
 ('/wiki/Richard_Rorty', 'Richard Rorty'),
 ('/wiki/Hilary_Putnam', 'Hilary Putnam'),
 ('/wiki/Neopragmatism', 'neopragmatists'),
 ('/wiki/Nicholas_Rescher', 'Nicholas Rescher'),
 ('/wiki/Robert_Brandom', 'Robert Brandom'),
 ('/wiki/Susan_Haack', 'Susan Haack'),
 ('/wiki/Cornel_West', 'Cornel West'),
 ('/wiki/Naturalized_epistemology', 'naturalized epistemology'),
 ('/wiki/W.V.O._Quine', 'W.V.O. Quine'),
 ('/wiki/Hilary_Kornblith', 'Hilary Kornblith'),
 ('/wiki/Contextualism', 'contextualism'),
 ('/wiki/Ontological', 'ontological'),
 ('/wiki/Social_constructivism', 'Constructivism'),
 ('/wiki/Knowledge', 'knowledge'),
 ('/wiki/Truth', 'truth'),
 ('/wiki/Idealism', 'Idealism'),
 ('/wiki/Ontological', 'ontological'),
 ('/wiki/Metaphysical', 'metaphysical'),
 ('/wiki/Transcendental_idealism', 'transcendental idealism'),
 ('/wiki/Bayesian_epistemology', 'Bayesian epistemology'),
 ('/wiki/Thomas_Bayes', 'Thomas Bayes'),
 ('/wiki/Subjective_probability', 'subjective probabilities'),
 ('/wiki/Probability_theory', 'probability theory'),
 ('/wiki/Rationality', 'rationality'),
 ('/wiki/Dutch_book', 'Dutch books'),
 ('/wiki/Feminist_epistemology', 'Feminist epistemology'),
 ('/wiki/Feminist_theory', 'feminist theory'),
 ('/wiki/Miranda_Fricker', 'Miranda Fricker'),
 ('/wiki/Sandra_Harding', 'Sandra Harding'),
 ('/wiki/Elizabeth_S._Anderson', 'Elizabeth Anderson'),
 ('/wiki/Social_epistemology', 'social epistemology'),
 ('/wiki/Boaventura_de_Sousa_Santos', 'Boaventura de Sousa Santos'),
 ('/wiki/Indian_philosophy', 'Indian schools of philosophy'),
 ('/wiki/Hindu_philosophy', 'Hindu'),
 ('/wiki/Nyaya', 'Nyaya'),
 ('/wiki/Carvaka', 'Carvaka'),
 ('/wiki/Jain_philosophy', 'Jain'),
 ('/wiki/Buddhist_philosophy', 'Buddhist'),
 ('/wiki/Carvaka', 'Carvaka'),
 ('/wiki/Materialists', 'materialists'),
 ('/wiki/Empiricists', 'empiricists'),
 ('/wiki/Aj%C3%B1ana', 'Ajñana'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Dharmakirti', 'Dharmakirti'),
 ('/wiki/Philosophical_skepticism', 'philosophical skepticism'),
 ('/wiki/Perspectivism', 'perspectivism'),
 ('/wiki/Jain_epistemology', 'Jain epistemology'),
 ('/wiki/Testimony', 'testimony'),
 ('/wiki/Decision_theory', 'decision theory'),
 ('/wiki/Logic', 'logic'),
 ('/wiki/Probability_theory', 'probability theory'),
 ('/wiki/Computability_theory', 'computability theory'),
 ('/wiki/Philosophy', 'philosophy'),
 ('/wiki/Computer_science', 'computer science'),
 ('/wiki/Economics', 'economics'),
 ('/wiki/Statistics', 'statistics'),
 ('/wiki/Metaphilosophical', 'metaphilosophical'),
 ('/wiki/Philosophical_method', 'methods')]
In [108]:
! ../.g

[master 9cbf06d] web
 1 file changed, 84 insertions(+), 182 deletions(-)
Counting objects: 4, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 1.64 KiB | 838.00 KiB/s, done.
Total 4 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/macbuse/macbuse.github.io.git
   3057a2c..9cbf06d  master -> master
In [ ]: