GSoC 2013
- Note to students
- References
-
Ideas
- Project GSoC1: Multiple database support, implement MongoDB Adapter
- Project GSoC2: Design a new Real Time Reports Dashboard
- Project GSoC3: Feed Analytics
- Project GSoC4: Reporting Visualization
- Project GSoC5: JavaScript improvement/CoffeeScript
- Project GSoC6: Feedback Automation
- Project GSoC7: Advanced Security
- Project GSoC8: Intelligent Hotkeys
All projects would start with a brainstorming session with the students and a few other team members (Adrian, Fabian, Matt) and think together about the ideal outcome of the projects that fits the student level and interest. There is a lot of potential for innovation, and discussing topics together will ensure we aim high while still being realistic and ship something great. Piwik is agile, with extensive tests, it is easy to contribute and innovate fast, and we can ship in a new beta version at any time, so you see your code used in production very fast.
For any requests, email [email protected] and we would be glad to assist or answer questions
Relevant skills:
- Technologies: PHP 5.3, Mysql
- Software design: architecture, design patterns
- QA: Unit/Integration testing, TDD.
Interest and keywords: Data analysis, Visualization, Big Data, Performance, Security, Software Design, Refactoring, Algorithms, Scalability.
- Coding Standards – Submitting pull requests? There are a few guidelines to apply for code clarity and maintainability.
- Development Process – Everything from pull requests submissions to the official release processes.
- Get started with Piwik development – Piwik Puppet Module + Vagrant.
- Contributing to Piwik with Git – Learn how to check out the Piwik source code, create your changes, and submit a pull request.
- Quality Assurance Tools – Quality assurance is automated by a fully-fledged environment. Find out how in this section.
Brief explanation: Implement a new Data Tracker and Data Archiver layers, so that we can plug in a new MongoDB Adapter.
Knowledge Prerequisite: PHP 5.3 is the main language used in Piwik. JavaScript and jQuery are used for visualization, but are not a key prerequisite. It is advantageous if you have worked with MySQL/MongoDB before.
Expected results
- Easier horizontal scalability
- Support the full Piwik feature set including Segmentation and “Real time” reports. http://piwik.org/docs/analytics-api/segmentation/ http://piwik.org/docs/real-time/
- Measurable code: Extend our existing load tests to test the new code performance.
List of tasks
- Test MongoDB prototype (visit/actions tables and write archiving code prototype)
- Refactor Piwik and introduce a layer in the code to allow a new database engine driver. Move the code to the Mysql Adapter.
- Write the new MongoDB adapter
- Check that unit tests + integration tests pass which would prove the new code is fully working
- Open Tasks:
- How can we support Segmentation, the ability to get all reports filtered to specific attributes?
- Support for "Real time" widget, and Visitors > Visitor Log feature allowing to browse through the logs, based on the timestamp and siteId
- Testing Performance under high load
- Document how to setup MongoDB for Piwik, known limitations, expected performance
- Involve Beta testers with high traffic websites (many piwik users are interested)
Notes
- Mysql could still be used for websites, users, permissions, reports, etc. The API code will be mostly unchanged and it makes this project reasonable and do-able during a Google Summer of Code.
- In this project we focus code changes on Tracking & Archiving only. Nosql/Mongodb could be used only for the equivalent data of piwik_log_* and piwik_archive_*
- Running Piwik at scale will require to run both Mysql and Nosql at the same time. Most page renders in Piwik will connect to both the Mysql (to select websites, users, permissions, session) and Nosql databases (to trigger archiving, or read aggregate reports).
- We would not investigate PostgreSQL support at this stage, but really focus on Tracking + Archiving layer rewritten in MongoDB. Supporting other databases will hopefully be a possibility after the project is finished.
Deadline Piwik 2.0 will likely be released around September/October just after the end of Google Summer of Code, so it would be great if any software API change is pushed to GIT by the end of the GSoC2013 by the students (with possibility to continue working and improving the adapters once we have finalized the API)
Brief explanation: The goal is to redesign the Real Time reporting to make it more useful, more informative. Plans are to show Top Pages, Top Actions (goal conversions, downloads), Top Referrers, Visitor Map, all at the same time on a new report. It would also refresh automatically every 10 seconds.
This project is more creative in terms of interface design and innovative possibilities in data visualization.
Mentor: Fabian or Matt or any other team member.
Knowledge Prerequisite: PHP 5.3 is the main language used in Piwik. JavaScript and jQuery are used for the frontend visualization.
Expected results
- New real-time dashboard
- Improved visualization of real-time data using more sophisticated widgets
Notes
This feature has been suggested dozens of times in the forums and in tickets in trac such as: http://dev.piwik.org/trac/ticket/2439
Brief explanation: Feed Analytics is one of the missing key features for Piwik. The goal for this project is to measure website feeds, track subscribers and new user clients.
Mentor: Fabian or Matt or any other team member.
Knowledge Prerequisite: PHP 5.3 is the main language used in Piwik. JavaScript and jQuery are used for the frontend visualization.
Expected results
- Users can add feeds to their websites and track them with Piwik
- Tracked data is displayed in the Piwik backend
- New metrics are available via the API (subscribers, reach, etc.)
Brief explanation: Piwik uses jqPlot to visualize data. jqPlot can be easily extended and new renderer added. For this project we want to explore new ways to visualize our data and help users better understand their traffic.
Mentor: Fabian or Matt or any other team member.
Knowledge Prerequisite: JavaScript and jQuery/jqPlot are used for the frontend visualization. You should also have some basic knowledge of HTML5 and CSS3.
Expected results
- New visualization options
- Customizable plots
Brief explanation: Parts of Piwik still use some plain JavaScript which should be refactored into jQuery plugins to ensure better re-usability and cleaner code. The piwik.js tracking script should be re-implemented in CoffeeScript. CoffeeScript will allow us to also have automated tests for all client-side code.
Mentor: Fabian or Matt or any other team member.
Knowledge Prerequisite: This project requires you to dig deep into the JavaScript code used in Piwik. Familiarity with jQuery certainly help.
Expected results
- Testable JavaScript/CoffeeScript code
- Reorganization of currently used JavaScript
Brief explanation: Allow users to automatically send feedback to a central feedback server whenever an exception occurs in the frontend and take a snapshot of the screen region.
Knowledge Prerequisite: This will require students to work on the client-side using JavaScript/CSS3 and the server-side using PHP5.3.
Expected results
- Detailed exception log
- Attachable Screenshot of a screen region (html2canvas / jsfeedback)
Notes
This feature gives us an insight on how people use Piwik and which versions run into trouble. Direct feedback will greatly improve user interaction with Piwik and allow us to analyze the problems that occur in the wild.
Brief explanation: Piwik is a very secure framework, but there are features that can improve the security even more. The new Password API
Knowledge Prerequisite: This project requires student to work with the latest PHP 5.5 API and backport it to 5.3.
Expected results
- Salted passwords using the Password API
- Smooth upgrade from currently unsalted password
- IP ban after N unsuccessful logins
Brief explanation: Implement Hotkeys to easily traverse through any Piwik installation.
Knowledge Prerequisite: This project will heavily focus on JavaScript and the Piwik API.
Expected results
- Easy access to key features of Piwik (Websites, Menus)
- Traversable menus with keyboard shortcuts