Analyzing Open Source development (part 3)

In last post about analyzing open source development I mentioned that this one would be about massaging people information to have unique identities for all the project contributors.

But before that, I would like to explore something different. How to get data from multiple repositories? What happens when I want data from a whole GitHub organization’s or user’s repositories?

The obvious answer would be:
1. Let’s get the list of repositories:


import requests

def github_git_repositories(orgName):
    query = "org:{}".format(orgName)
    page = 1
    repos = []
    
    r = requests.get('https://api.github.com/search/repositories?q={}&page={}'.format(query, page))
    items = r.json()['items']
    
    while len(items) > 0:
        for item in items:
            repos.append(item['clone_url'])
        page += 1
        r = requests.get('https://api.github.com/search/repositories?q={}&page={}'.format(query, page))
        items = r.json()['items']
    
    return repos

2. And now, for each repository, run the code seen in previous post to get a dataframe for each one in list and concat them with:


df = pd.concat(dataframes)

For organizations or users with a few repositories, it would work. But for those with hundreds of repositories, how long would it take to go one by one fetching and extracting info?

Would there be a fastest approach? Let’s play with threads and queues…
Continue reading “Analyzing Open Source development (part 3)”

Analyzing Open Source development (part 1)

Simple analysis of open source development in public administrations can be done very easily. This post describes the initial steps to understand how to obtain previous post results.

We’ll learn how to use Perceval. It’s the tool responsible for data retrieval in GrimoireLab, the free, open source software framework for software development analytics.

Take some coffee or tee, and let’s start!

Laptop, notebook and coffee

Continue reading “Analyzing Open Source development (part 1)”

PyMaemo en Fremantle

Ayer se añadió el soporte para PyMaemo en Fremantle (instrucciones de instalacion), y tuve que probarlo…

  1. PyMaemo instalado
  2. sympy descargado

  3. [sbox-FREMANTLE_X86: ~] > python2.5
    Python 2.5.2 (r252:60911, Mar 27 2009, 18:30:12)
    [GCC 4.2.1] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from sympy import *
    >>> x,y,z = symbols('xyz')
    >>> Plot(x*y**3-y*x**3)

Y aquí está el resultado:

Sympy 3D Plot en Fremantle

¿Quién quiere crear una calculadora científica avanzada para Maemo 5? 🙂

[tags]maemo, python, sympy, fremantle[/tags]

Ubuntu Mobile e Intel

Durante estos días se ha celebrado el Intel Developer Forum (IDF), y además de presentar algunos prototipos de nuevos dispositivos móviles, se ha hablado de Ubuntu Mobile y MobLin, así que es recomendable echarle un vistazo a las presentaciones hechas.

En cuanto a Ubuntu Mobile, la distribución avanza, y para los interesados en desarrollar aplicaciones para ella, es recomendable mirar la guía disponible, y ver lo importante que va a ser Python en esta versión.

Por cierto, hablando de cacharros con Linux, la GP2X va a ser actualizada, añadiendo pantalla táctil. Más información en su página web.

Technorati: ,