Purdue ECE students develop spoken IDEs for Python in Natural Language Processing competition
Students enrolled in ECE 49595NL: Natural Language Processing in Purdue University’s Elmore Family School of Electrical and Computer Engineering recently undertook a rigorous challenge to develop a Spoken Integrated Development Environment (IDE) for Python. This system allowed users to write, execute, edit, and debug Python code exclusively through spoken commands, without any physical interaction with the computer.
As part of this assignment, students were given access to Microsoft Azure Speech-to-Text and Text-to-Speech APIs, as well as the Microsoft Azure OpenAI ChatGPT API, to implement their solutions. Each team had full autonomy in designing their approach, leading to a variety of innovative implementations. The project culminated in a competitive evaluation, where student teams were tested on their system’s efficiency and accuracy in executing programming tasks.
Natural Language Processing (NLP) is a subfield of artificial intelligence that enables computers to process and interpret human language. It is widely applied in speech recognition, machine translation, sentiment analysis, information retrieval, and conversational AI. The competition in ECE 49595NL demonstrated the capabilities of NLP in software development by enabling students to create speech-driven coding environments, highlighting the rapid advancements in AI-assisted programming.
Under the guidance of Jeff Siskind, Professor of ECE, 13 teams, each consisting of two to four students, developed spoken IDEs capable of generating Python code from natural language task descriptions. Some teams extended their functionality to allow for code modifications and debugging via voice commands.
To assess the effectiveness of these systems, a timed competition was conducted. Each team was assigned a programming task:
Write a program that counts the number of vowels in the string "West Lafayette" (excluding "y").
Teams were ranked based on completion time, with grading and prizes awarded accordingly. All 13 teams successfully completed the task within 90 seconds, with the fastest team achieving a completion time of 12.65 seconds.
Following the initial timed assessment, each team was given a series of additional coding challenges to further evaluate the robustness of their system. These tasks included:
- Printing the nth Fibonacci number
- Sorting and manipulating lists
- Determining whether a number is prime
- Computing the Euclidean distance between two lists
- Checking if a string is a palindrome
While not all teams were able to complete every task, all teams successfully solved multiple problems, demonstrating the practical feasibility of voice-driven programming interfaces.
ECE student Ishan Kumar was a member of the team that finished in first place. He says the competition taught his team a lot about using Large Language Models (LLMs) in coding projects.
“Originally, we thought LLMs like ChatGPT were mainly used for the individual where you can ask it questions similar to Google for quick answers,” says Kumar. “Working on this project showed us the potential that LLMs have beyond just answering questions. It has given us a new mindset for the possibilities LLMs have in professional workflows and projects.”
Reflecting on the competition, Professor Siskind emphasized the remarkable progress in NLP and speech recognition, particularly in comparison to earlier efforts in spoken programming.
“I was impressed by the results. This would have been infeasible just two years ago. Similar research was conducted in the 1970s and 1980s, but those systems were not successful,” said Siskind.” During my graduate studies in the 1980s, projects such as the MIT AI Lab’s ‘Programmer’s Apprentice’ attempted to generate and edit code from natural language input. None of these efforts yielded functional systems. The fact that this task can now be assigned as an undergraduate homework project is a testament to the rapid advancements in NLP and AI.”
This competition underscores the growing impact of AI-driven programming tools in modern software development. Tools such as GitHub Copilot and ChatGPT are already assisting developers by suggesting code, debugging errors, and automating repetitive tasks. The successful implementation of voice-controlled programming environments in this competition suggests that similar hands-free coding solutions may become increasingly viable in professional software development.
The rapid progress in speech recognition and NLP continues to push the boundaries of human-computer interaction. Through initiatives such as this, Purdue ECE remains at the forefront of cutting-edge AI and engineering education, equipping students with the knowledge and skills to advance the field of computational intelligence.
Find more photos here.