Project Details: GSoC 2017 with phpMyAdmin

As I had posted earlier, my proposal for ‘Improving phpMyAdmin’s Selenium testsuite and Error Reporting Server’ got accepted in Google Summer of Code with phpMyAdmin.

The project aims to provide some added and improved functionality in the areas of functional/selenium testing and phpMyAdmin’s error reporting server.

The project details are presented under two broad headings:

  1. Tasks related to Error Reporting Server
  2. Tasks related to Selenium testing

Tasks related to Error Reporting Server

This involves implementing these tasks over the summer:

  1. Issue #98: Follow Github issue state
  • Current behaviour: No such option. Prior to migration of our issue tracker, the state of reports were synced with the linked SourceForge issues. On migration, this functionality has been lost.
  • Expected behaviour: The error reporting server should be able to follow state of linked issues and update state of the reports linked to that issue according to them.
  • Implementation Procedure:
    1. Github provides event webhooks for any repository which can be listened by a controller on our reporting server.
    2. Once an issue (close) event is received, the controller will set the state of all the reports linked with that issue to ‘closed’. This way we would not have to run a cron job, and this would ideally be tracking changes in real time.
    3. If the operation takes a lot of time, we might have to use queueing mechanisms to hold these event payloads received from Github.
    4. Security aspects have to be considered as mentioned here
  • Alternate Implementation Procedure:
    1. Github developer APIs provides a rich way of interacting with the issues on a repository. We could use the APIs provided, for example:
      GET /repos/:owner/:repo/issues/:number
      to get the current state of the linked issue for a report and update the same on server
    2. This can be implemented as a shell, which could be run as a cron-job using the console tool that CakePHP provides.

      2. Issue #31 : Provide email notification for new reports

  • Current Behaviour: New reports are not reported to the developers via emails. This leads to the developer manually checking the error reporting server to check, if new relevant reports have been added.
  • Expected behaviour: New report generation would be accompanied by emailing a small summary of the new report’s details to the developer community (maybe through a new ‘bugs’ mailing list).
  • Implementation Procedure:
    1. Cakephp3 has a core library included for custom emails through cakephp. The ReportsController.php, in its creation of the report, would also include a function call to mail the summary of new report’s details to the bugs mailing list.
      Reference – http://book.cakephp.org/3.0/en/core-libraries/email.html

      3. Issue #106 : Notifications handling

  • Current Behaviour: We don’t provide any option to clear all notifications. Moreover, there is not even a ‘Select all’ checkbox to select all the notifications on a page.
  • Expected behaviour: The missing ‘Clear all notifications’ button (and the corresponding action) should be provided so that the developer can start with a clean slate. Moreover, a ‘Check all’ checkbox would enable the developers to quickly filter and clear the notifications shown currently on the page
  • Implementation Procedure:
    1. We use Data tables to populate, order and enable search queries (though order and search are actually run in with SQL queries) in the tables on Notifications page.
    2. A check-all box can be added similar to what is present on the reports page
    3. The ‘Clear all notification button can added above the table header (may be right-aligned in the same row as ‘Action for Selected Notifications’)

      4. Issue #119 : Improve generated issues content

  • Current Behaviour: Once a report on the error reporting server is linked to an issue, a comment is posted with the error type, error message, exception type and the link to the report.
  • Expected behaviour: It would be really help the developers looking at the comment in the issue tracker if affected phpMyAdmin version, script name and number of incidents are also included in the generated comment.
  • Implementation Procedure:
    1. The changes have to made in the src/Controller/GithubController.php file
    2. We would have to fetch the required information related to the report from the database using appropriate models.
    3. This extra information can be included in the data being posted in the request to the Github server while posting the comment (while linking to existing issue) or while creating a new issue.

      5. Issue #120 : Simplify Issue states

  • Current Behaviour: Since we used to track the issue state from SourceForge through a cron job, we had adapted to the issue states available in SourceForge’s issue tracker and had added corresponding issue state for our reports.
  • Expected behaviour: Since we have moved to Github issue tracker, we would need only three states namely: opened, closed and forwarded. Opened is the default when a new report is generated, it is set to forwarded when the report is linked to a new or existing issue. Once an issue gets closed on Github, the linked reports are also marked as closed.
  • Implementation Procedure:
    1. The changes would be involved to the $state array in the src/Model/ReportsTable.php
    2. Then changes would be required in the flow which creates a new issue and/or links a report to an existing issue on Github, so that the state of the linked report can be changed to forwarded.
    3. The other change required would be that the state of the report should be updated once we receive a issue-closed event from the Github webhook.

      6. Issue #123 : Allow search by filename

  • Current Behaviour: We allow the search in data tables on reports page based on exception name, message, phpMyAdmin version affected, state, exception type.
  • Expected behaviour: The search functionality does not help much when the exception name and the message are very similar but are actually present in different files. We should have a column stating the filename and allow search by that column to help distinguish such reports.
  • Implementation Procedure:
    1. Adding a column involves changes to the template, the view action in src/Controller/ReportsController.php by changing the $aColumns array.
    2. Moreover, the searchable property for this newly added column in the data table would be automatically enabled. (It can be disabled by specifying in the webroot/js/custom.js aoColumnDefs field, but we don’t have to touch it in this case)

      7. Issue #74 : Read-only public interface

  • Current Behaviour: For accessing the error reporting server, one needs to have commit access to the phpmyadmin/phpmyadmin repository on Github. This prevents contributors (non-team members) to access the application. Currently, any issue on the tracker that has been forwarded from a report on error reporting server might be incomprehensible (or at least a pain to work on a fix for) to any non-team developer, since (s)he can’t even take a look at the actual report/incidents.
  • Expected behaviour: We should allow for public read-only interface so that anyone can take a look at the error reports. This would enable democratization of the technology and help in increasing the developer engagement in the community.
  • Implementation Procedure:
    1. The reports main page (i.e. the index action in ReportsController) can remain as it is, while the view action can be changed to have the action buttons like ‘Mark same as’, ‘Create new issue’, ‘Link to an existing issue’ made conditional on whether the user is logged in (ideally only team members)
    2. The currently unused function ‘canCommitTo’ in the Github API can be used to check whether the user is authorized to access the report actions. Moreover, the $whitelist in src/AppController.php would have to be altered to allow for anonymous users to access the read-only interface

      8. Issue #129 : Use cleaner alternative syntax for control structures in View templates

  • Current Behaviour: The templates, in the current code, use the standard syntax for the control structures that is used in the .php files. It makes it very inconvenient to read and comprehend the code, since there are a lot of braces and they may not be correctly indented etc.
  • Expected behaviour: Use alternative syntax in template files
  • Implementation Procedure:
    1. Rewrite the control structures in the template files using the alternative syntax, of course, without breaking any existing functionality.

Tasks related to Selenium testing

These tasks are broadly divided into 2 major sub-lists:

  1. Fixing existing tests: This involves fixing the existing set of broken selenium tests. This would help in making the overall test suite reliable, so that it can be run on every  commit.

 

S. No.

Test name Current status
1 CreateDropDatabaseTest Works
2 CreateRemoveUserTest Works
3 DbEventsTest Inconsistent
4 DbOperationsTest Broken
5 DbProceduresTest Broken
6 DbStructureTest Broken
7 DbTriggersTest Broken
8 ExportTest Broken
9 LoginTest Works
10 NormalizationTest Broken
11 PrivilegesTest Broken
12 ServerSettingsTest Broken
13 TableBrowseTest Broken
14 TableCreateTest Broken
15 TableInsertTest Broken
16 TableOperationsTest Broken
17 TableStructureTest Broken
18 TableTrackingTest Broken
19 XSSTest Broken
20 ImportTest Broken

 

Assuming that each test and its test-cases can be fixed in a day’s work (on an average), fixing all the current tests would require 3 weeks of time.

Adding new tests: The selenium testsuite will be extended to common operations by adding a new set of tests and improve the selenium testsuite coverage.

S. No. Feature Covered Expected duration
1 Typing and executing SQL query – Server SQL 1-2 day(s)
2 Typing and executing SQL query – Database SQL 1-2 day(s)
3 Typing and executing SQL query – Table SQL 1-2 day(s)
4 Granting an user access to a database 1-2 day(s)
5 Import tests 1-2 week(s)
6 Exports tests (expand to test more options, for Server-level, Db-level, table-level) 4 days

 

I would be posting weekly updates every Monday, about the work undertaken during the previous week as soon as the coding period starts.

Looking forward to another exciting summer with phpMyAdmin. 🙂

Advertisements

About Deven Bansod

I am a recent graduate with a dual degree in B.E.(Hons.) Computer Science Engg. and M.Sc.(Hons.) Economics from BITS Pilani, Pilani (India). I am interested in and have been contributing to development of free and open source software s (FOSS). More recently, I have been contributing to phpMyAdmin, a web interface to MySQL, written in PHP. I'm looking for opportunities to contribute to interesting open-source softwares.

Posted on May 14, 2017, in GSoC 2017, phpMyAdmin and tagged , , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: