In this post I show how to enrich a ggplot map with data obtained from the Open Street Map (OSM) API. After adding elevation details to the map, I add water bodies and elements identifying human activity. To highlight the areas more densely inhabitated, I propose to use a density-based clustering algorithm of OSM features.
Thursday, 9 August 2018
In Medieval Europe, information was physically concentrated in very few secluded libraries and archives. Powerful institutions managed them and regulated who could access what. The library of the fictional abbey that is described in Umberto Eco’s The Name of the Rose is located in a fortified tower and only the librarian knows how to navigate its mysteries. Monasteries played an essential role in preserving written information and creating new intelligence from that knowledge. But being written information a scarce resource, with the keys to libraries came also authority and power. Similarly, Internet companies are amassing information within their fortified walls. In so doing, they provide services that we now see as essential but they also contravene the two core principles of the Internet: openness and decentralisation.
Monday, 7 May 2018
The abundance of economic data and the scarcity of social data with a comparable level of granularity is a problem for the quantitative analysis of social phenomena. I argue that this fundamental problem has misguided the analysis of the electoral results of the Five Star Movement (M5S) and its interpretation. In this article, I provide statistical evidence suggesting that — in the South — unemployment is not associated with the exceptional increase in the M5S support and that local participation is a stronger predictor of support than most of the demographics.
The 2018 Italian general elections (elections, since both the Chamber of Deputies and the Senate, were renewed) saw
- a significant increase in the number of votes for two parties, the Five Start Movement (M5S) and the League (formerly Northern League),
- an increase in the importance geography as an explanatory dimension for the distribution of votes.
The following two maps show where the M5S and the League have increased electoral support from 2013 to 2018. (Electoral data are always data for the election of the Chamber of Deputies).
The geographic pattern is quite simple. The M5S has increased its support in the South and maintained its votes in the North, the League has significantly strengthened its support in the North but has also collected votes in the South, where it had virtually no support. The third and the fourth most voted parties, the Democratic Party (PD) and Berlusconi’s Forza Italia (FI), have lost votes almost everywhere. If we map the results of the four parties side-by-side with the same scale, the PD and FI almost faded into the background.
Yet, major metropolitan areas do not always follow the national trend. If Naples unambiguously voted M5S, Turin, Milan and Rome did saw the Democratic Party as the most voted party in the wealthiest districts.
The territoriality of the results, especially along the North-South dimension, makes the analysis especially complicated. This because the strong result of the League in the North and of the M5S in the South might simplistically suggest that immigration (which is much stronger in the North) explains the League’s result in the North and unemployment and poverty (stronger in the South) explain the M5S’s result in the South. This reading is especially attractive since immigration and the M5S proposal to introduce a guaranteed minim income have dominated the campaign.
Tuesday, 20 March 2018
This article describes the simulation behind the app that you find here
This simulation of the results for the 2018 general election is based on the results from the last two national elections (the Italian parliament election in 2013 and the European Parliament election 2014) and national polls conducted until 16 February 2018. The simulation is based on one assumption, which is reasonable but not necessarily realistic: the relative territorial strength of parties is stable. From this assumption derives that if the national support for a party (as measured by national voting intention polls) varies, it varies consistently and proportionally everywhere. A rising tide lifts all boats and vice versa. The assumption has some empirical justification. If we compare the difference from the national support (in percentage) for each district in 2013 and 2014 we see a significant correlation, especially in the major parties.
Tuesday, 27 February 2018
The 2016 Italian referendum torpedoed the constitutional reform presented by the government presided by Matteo Renzi (41). According to the final count, which includes 1.2 million votes cast overseas, the reform was rejected by almost 60% of the voters.
Three parties played a predominant role during the electoral campaign: the ruling Democraric Party (PD), leaded by the chief of government Renzi, the Five Star Movement (M5S), founded and leaded by Beppe Grillo (68), and the Lega Nord (LN), leaded by Matteo Salvini (43). The fourth Italian party, Forza Italia, for different reasons – including the health of Silvio Berlusconi (80) – played a minor role.
Monday, 5 December 2016
Leggo e rispondo al post di Massimo Mantellini (Il M5S, il wifi e il principio di precauzione) in cui si evidenzia con preoccupazione come il Movimento abbia portato in Parlamento, dunque in qualche modo legittimandole, posizioni anti-scientifiche; un “pensiero tossico, banale e a suo modo inattaccabile, che nuoce al Paese intero”.
Il Movimento Cinque Stelle con un bacino elettorale che si aggira tra il 25 e il 30% (8.5-10 milioni di persone) è necessariamente complesso in termini di rappresentanza demografica e di diversità di opinione. Considerando un astensionismo del 25%, se vi trovate in fila al supermercato delle 10 persone che vi precedono circa due votano M5S. Purtroppo questa complessità raramente traspare nelle narrazioni giornalistiche, e chi fa informazione tende (troppo) spesso a preferire i tratti caricaturali (da cappello di carta stagnola o da gita in Corea del Nord, per intenderci). Ma questo tipo di informazione è sbagliata: primo perché distorce nella semplificazione, secondo perché incoraggia comportamenti macchiettistici, grotteschi e sbracati da parte di chi sedendo in istituzioni affollate cerca visibilità.
Friday, 22 July 2016
The Five Star Movement (M5S) obtained two major victories in the second round of municipal elections on 19 June 2016 in Rome and Turin. Rome attracted the most international attention but it is M5S’ victory in Turin that is likely the most consequential for them and other European anti-establishment parties.
In Rome, a municipality with 2.8 million people and an annual budget of €5 billon, Virginia Raggi (age 37) gained doubled the votes of her contender Roberto Giachetti (age 55). In Turin, a city with a population of 900,000 and an annual budget of €1.69 billion, Chiara Appendino (age 31) outstripped Piero Fassino (age 66) by about 10 percentage points.
Continue reading on Pop Politics Aus
Friday, 8 July 2016
Explicit semantic analysis (ESA) was proposed by Gabrilovich and Markovitch (2007) to compute a document position in a high-dimensional concept space. At the core, the technique compares the terms of the input document with the terms of documents describing the concepts estimating the relatedness of the document to each concept. In spatial terms if I know the relative distance of the input document from meaningful concepts (e.g. ‘car’, ‘Leonardo da Vinci’, ‘poverty’, ‘electricity’), I can infer the meaning of the document relatively to explicitly defined concepts because of the document’s position in the concept space.
Tuesday, 26 April 2016
To talk about identity and soul of the Five Star Movement (M5S) is not only politically contentious but also practically challenging because of the different axes (at least three) along which the M5S has been developing: the vertical top-down axis from Beppe Grillo to his followers (and sympathising voters), the horizontal axis connecting thousands of militants across the country to local, flexible and loosely organised meetups, and finally the cloudy axis linking Internet users through the different online communicative platforms pertaining to the Movement. The academic literature and the media have been prevalently interested in mapping the provenance of votes. I will try here to show some data also on the position of the M5S derived from its 2013 electoral program and the political background of both the onsite and online activists of the Movement.
But let’s first start briefly introducing the trajectory of a movement that vehemently refuses to be called a party or to be associated with any traditional political identity.
Continue reading on the blog of the WZB.
Tuesday, 12 May 2015
The Normalized Difference Vegetation Index (NDVI) estimates the greenness of plants covering the surface of the Earth by measuring the light reflected by the vegetation into space. The main idea behind the NDVI is that visible and near-infrared light is absorbed in different proportions by healthy and unhealthy plants: a green plant will reflect 50% of the near infrared-light it receives and only 8% of the visible light while an unhealthy plant will reflect respectively 40% and 30%. NDVI can then be used to quantitatively compare vegetation conditions across time and space (and indeed is quite widely used, a Google Scholar search on NDVI produced 60,500 hits).
Thursday, 14 February 2013
- @alessabocchi 2.6 is the deficit, not the debt. The French debt-gdp ratio is 97%, the Italian debt-gdp ratio is 130… https://t.co/xvexdy9UQg
- @lmisculin Si, il dato di novembre 2018 é strano. O il campione é cambiato radicalmente e gli amministratori non so… https://t.co/4kveKBuQ6m
- @lmisculin Un aumento del genere in così poco tempo fa sospettare che l’eurobarometro abbia dei problemi di pesi. M… https://t.co/nw1ARAJhaD
- RT @TPpodcast_: Start the week with a round of applause for listener @FrBailo, who did the legwork to back up David’s hypothesis on the age…
- @TPpodcast_ Thanks, I was so inspired by this episode that I actually mapped the trend in the median age for parlia… https://t.co/hDuQEo8zom
- But the largest drop is not surprisingly with the election of 2013 when the median age dropped 9 years from 55 to 4… https://t.co/Jn3h9lnkiA
- The @Mov5Stelle, of course, has significant responsibilities for bringing down the median - which, in any case, is… https://t.co/yaz3iowhpK
- In the last episode of @TPpodcast_, David Runciman notes that with Italy's election of 2018 the median age of MPs p… https://t.co/hSEVF7JJLM
- @CaterinaFroio @_itanes @PSA_IPSG No, mi spiace. Mi viene in mente solo l'Eurobarometro. Ma con lo stesso problema… https://t.co/2vgzLaO1U5
- @CaterinaFroio @_itanes @PSA_IPSG No, credo di no. Ma la documentazione é abbastanza dettagliata. Credo che in qual… https://t.co/dHkHy4jas8
- Day 12 – little helper diveThis little helper is a debug function if you do not work with RStudio. Der Beitrag Day 12 – little helper dive erschien zuerst auf STATWORX.
- Reading List Faster With parallel, doParallel, and pbapplyI have several tables that I would like to load as a sole data frame. Derived functions from read. table () have a lot of convenient features, but it seems like there is a lot of steps in the implementation that would slow things down. The gain in pe...
- Sharing Modeling Pipelines in RReusable modeling pipelines are a practical idea that gets re-developed many times in many contexts. wrapr supplies a particularly powerful pipeline notation, and a pipe-stage re-use system (notes here). We will demonstrate this with the vtreat data preparation system. Our example task is to fit a model on some arbitrary data. Our model will try […]
- Le Monde puzzle [#1075]A new Le Monde mathematical puzzle in the digit category: Find the largest number such that each of its internal digits is strictly less than the average of its two neighbours. Same question when all digits differ. For instance, n=96433469 is such a number. When trying pure brute force (with the usual integer2digits function!) le=solz=3 […]
- Scraping NBA game data from basketball-reference.comI’m a casual NBA fan: I don’t have time to watch the games but enjoy viewing the highlights on Instagram/Youtube (especially Shaqtin’ A Fool!); I sometimes read game articles and analyses (e.g. Blogtable). Apart from the game being an amazing … Continue reading →
- The Role of Theory in Data AnalysisIn data analysis, we make use of a lot of theory, whether we like to admit it or not. In a traditional statistical training, things like the central limit theorem and the law of large numbers (and their many variations) are deeply baked into our heads. I probably use the central limit theorem everyday in […]
- The role of academia in data science educationI was recently asked to moderate an academic panel on the role of universities in training the data science workforce. I preceded each question with opinionated introductions which I have fused into this blog post. These are weakly held opinions so please consider commenting if you disagree with anything. To discuss data science education we […]
- Guest Post: Galin Jones on criteria for promotion and tenture in (bio)statistics departmentsEditor’s Note: I attended an ASA Chair’s meeting and spoke about ways we could support junior faculty in data science. After giving my talk Galin Jones, Professor and Director of Statistics at University of Minnesota, and I had an interesting conversation about how they had changed their promotion criteria in response to a faculty candidate […]
- “Do you have any recommendations for useful priors when datasets are small?”A statistician who works in the pharmaceutical industry writes: I just read your paper (with Dan Simpson and Mike Betancourt) “The Prior Can Often Only Be Understood in the Context of the Likelihood” and I find it refreshing to read that “the practical utility of a prior distribution within a given analysis then depends critically […]
- Prior distributions for covariance matricesSomeone sent me a question regarding the inverse-Wishart prior distribution for covariance matrix, as it is the default in some software he was using. Inverse-Wishart does not make sense for prior distribution; it has problems because the shape and scale are tangled. See this paper, “Visualizing Distributions of Covariance Matrices,” by Tomoki Tokuda, Ben Goodrich, […]
- Should we be concerned about MRP estimates being used in later analyses? Maybe. I recommend checking using fake-data simulation.Someone sent in a question (see below). I asked if I could post the question and my reply on blog, and the person responded: Absolutely, but please withhold my name because this is becoming a touchy issue within my department. The boldface was in the original. I get this a lot. There seems to be […]