6.5 C
New York
Wednesday, June 23, 2021

Github wants to improve code search with competition

Apparently following on from Imagenet, code-host Github wants to establish a deep learning competition for the semantic search in code with Codesearchnet. The dataset contains 6 million methods, some with documentation and metadata.

Must Read

Kamal Saini
Kamal S. has been Journalist and Writer for Business, Hardware and Gadgets at Revyuh.com since 2018. He deals with B2b, Funding, Blockchain, Law, IT security, privacy, surveillance, digital self-defense and network policy. As part of his studies of political science, sociology and law, he researched the impact of technology on human coexistence. Email: kamal (at) revyuh (dot) com

In the Imagenet project, researchers collect millions of photos that are assigned to specific categories. This means that deep-learning systems for image recognition can not only be trained but also compare their quality very well because of the same data. The code hoster Github now obviously wants to adapt this concept for the semantic search in source code and starts the project Code Calculations.

The announcement by Github states that source code search engines are often frustrating and never fully understand what they are asking for. And despite improvements in technology through the use of modern machine learning approaches, there has been a lack of a consistent set of data to evaluate the results. That’s exactly what codes should do.

The data set created by Github feeds from the code of open source projects on the platform of the provider and includes functions with documentation of the languages ​​Go, Java, JavaScript, PHP, Python and Ruby. For preprocessing the code data, Github relies on its own parser generator Tree-sitter and its function parser, which is used to generate ASTs and any documentation and metadata about the individual functions.

Github provides the dataset for download in an Amazon S3 bucket. Overall, according to the provider, this involves around six million methods, of which two million have associated documentation. There are also metadata such as the location of the code. The code for the created model is of course also found Github. Further details are described in a scientific paper.

Via | Github

- Advertisement -


Please enter your comment!
Please enter your name here

- Advertisement -

Latest News

Study reveals a vegetable that reduces risk of hypertension

Boiled or baked potatoes can help lower blood pressure, and French fries can be part of a healthy diet,...
- Advertisement -

More Articles Like This

- Advertisement -