Commit 76fbfa62 authored by Simone Aonzo's avatar Simone Aonzo
Browse files

S5 project proposal

parent 2e9713ae
......@@ -75,7 +75,7 @@ Students will become experienced with languages dedicated to digital audio proce
- Semester: S5
- Proposer: Renaud
- Description: Attach a camera to the platform, and use it to deliver colour-oriented audio description of scenes ("large dark red square patch in the bottom left, white diagonal stripe from top left to bottom right"...).
- Involved courses/skills/knowledges:
- Involved courses/skills/knowledge:
* BasicOS: modify the camera device driver to adapt it to the project
* IntroArchi: optimize the camera device driver with some assembly coding of performance-critical sections
* SoundProc: speech synthesis and speech recognition (to command the device with voice)
......@@ -90,7 +90,7 @@ Students will become experienced with languages dedicated to digital audio proce
- Semester: S5
- Proposer: Chiara and Max
- Description: Navigating an obstacle course blindfolded, guided by audio feedback from video analysis of the scene. The raspberry pi is connected to a camera and stereo headphones. The device is worn around the neck and captures the scene via the camera. The algorithm developed by the students has to detect the perimeter of the path and obstacles and guide the navigation of the blind person. The video input is transformed into audio feedback in the headphones to indicate the direction in which the person should walk. Students are free to choose the most appropriate algorithm for this purpose.
- Involved courses/skills/knowledges:
- Involved courses/skills/knowledge:
* BasicOS: modify the camera device driver to adapt it to the project
* IntroArchi: optimize the camera device driver with some assembly coding of performance-critical sections
* SoundProc: a stereo audio signal is used to direct the person through the obstacle course. Directionality exploits the spatial perception of our auditory system. The audio signal might have a higher frequency component or a higher loudness on the side towards which the person has to turn. More refined techniques such as binaural auralisation could be used. In this case, the direction is encoded as a synthesised acoustic event and placed in three-dimensional space.
......@@ -108,7 +108,7 @@ Students will become experienced with languages dedicated to digital audio proce
- Description:
* Option 1) Hand proximity recognition with proximity sensor to change volume (e.g. low volume to high volume). Hand gesture recognition using a camera, N classes = different gestures (determine sound characteristics, e.g. pitch and timbre).
* Option 2) There is a canvas (e.g. whiteboard) where adding objects or symbols changes the sound characteristics, e.g. pitch, timbre and loudness, and the colour and intensity of the lights.
- Involved courses/skills/knowledges:
- Involved courses/skills/knowledge:
* BasicOS: modify the camera device driver to adapt it to the project
* IntroArchi: optimize the camera device driver with some assembly coding of performance-critical sections
* SoundProc: Option 1: the sound changes according to the proximity of one hand and the gesture of the other hand; Option2: placing symbols or objects on the canvas corresponds to selecting different sounds.
......@@ -124,7 +124,7 @@ Students will become experienced with languages dedicated to digital audio proce
- Semester: S5
- Proposer: Pasquale
- Description: Realise a game in which 2 different devices (the chosen platform) play one against the other in turn. The game rules can be chosen in order to maxise the number of involved skills. E.g. Tetris (in turn, collaborative or competitive) or Pong, driven by hand gestures. E.g.2 Pictionary, the answer require character/speech recognition
- Involved courses/skills/knowledges:
- Involved courses/skills/knowledge:
* BasicOS: modify the camera device driver to adapt it to the project
* IntroArchi: optimize the camera device driver with some assembly coding of performance-critical sections
* SoundProc: the sound changes according on how in danger/close to the solution you are in the game
......@@ -135,4 +135,20 @@ Students will become experienced with languages dedicated to digital audio proce
- Target platform: Raspberry Pi 4, camera, speakers, eventually mic
### Consensus algorithm for image classification using a mesh network
- Semester: S5
- Proposer: Simone
- Vision statement: United we stand, divided we fall
- Description: in this project, students must configure their devices in order to create a mesh network. Then, they have to agree on a protocol to communicate in broadcast to which category belongs the animal (for example) that they detect through the webcam according to their image processing program. Then, using a consensus algorithm, decide "definitively" to which category it belongs. The logs must be in a predetermined format for evaluation.
- Involved courses/skills/knowledge:
* BasicOS and ComLab: config of the mesh network
* ImProc: image processing to detect and classify the animal
* ComProg: matching algorithm, programming
* IntroNet1: communication protocol
- Evaluation: We decide the "evaluation day" and we display some images on a big screen. Then, given a metric (like F1 score), each group will be evaluated on both the image classification of their image processing program, and the classification of their consensus algorithm.
- Target platform: Raspberry Pi 4 and camera
<!-- vim: set tabstop=4 softtabstop=4 shiftwidth=4 noexpandtab textwidth=0: -->
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment