Last week I’ve been working on admin review feature to incorporate the new changes happened in the user end.
With the addition of in-place editing feature, admins can now approve minor edits by actually looking the difference. They also have options to replace author/publisher if the previously set author was different.
Along with that, we now have an option to upload book cover. It can be found in ‘Add new book’ form. In the long run, it will be also appearing under each book cover(in book page) where user can upload a cover just like they edit other information.
In Week 9, I rewrote the complete crowdsourcing functionality of Grandham from a submission based approach to micro-edits based approach. Earlier we used to store the entire snapshot while recording modifications but now we store only the ‘edit’ (similar to diff, but in a structured database table) and pass it to admins for reviewing. This approach has some significant advantages including the easy generation of history of book pages.
In-place editing was integrated during the same week. Now participants can click on a field and edit it.
Last weekly I mainly worked on Search feature and admin interfaces. Here’s a quick review:
- Experimented with various search engines like Sphinx, Ferret, etc and finally decided to use Solr with sunspot gem as it worked well with unicorn data without any configuration.
- Wrote a general search backend interface for Grandham. We can plugin other search engines anytime as long as it communicated the way Grandham needs it.
- Added Admin interfaces managing Authors and Publishers.
I couldn’t post the weekly updates regularly for the last two times since I was kept more busy than I had expected and the festive holiday that came in between. Here’s a quick review of what happened during the 7th week of GSoC.
- Grandham got an UI revamp!
- Integrated new theme.
- Separate books / authors / publishers lists were implemented
- Updated user sign in / sign up pages
- Integrated database clear and fixed failing specs (It’s green now, yay!)
This week I’ve been mostly fixing bugs, refactoring code and building admin features. Grandham has built in checks to avoid duplication in Authors/Publishers now.
Submission Review Interface
We have a new interface this week – A screen to review crowdsourced information submissions. Administrators can use this feature to control the information displayed to user. Currently every review item has 3 actions – Approve, Archive or Comment. Out of these, ‘Comment’ is a new feature which needs to be implemented. It will be helpful for the task force to generate discussions around specific submissions.
After a couple of rounds of refactoring, Grandham got a Code Climate score of 4.0 from 1.61!
Last week, I implemented different parts of the application including the basics of user authorization. Also did a work on refactoring the code base, our tests are clean now.
We now have a page to display information of a specific book, which can be navigated from the books list. This page will eventually contain all the information, including location information and links to download the entire MARC data.
User authorization has been added to Grandham using the gem ‘devise’. Devise provides beautiful forms required for various user-authorization needs, I customized them to work with the look and feel for Grandham. Some of them are here:
(Social login needs to be implemented in coming weeks)
Our test suite was starting to clutter and due to a problem in factories the CI has been failing randomly. I’ve refactored the test suite and now it all works fine.
In other news, we have switched the browser to run tests from selenium – firefox to poltergeist – phantomjs. It helps us to run tests faster. Also I’ve updated our API and data import script.
Week 2 of Grandham was filled with more architecture decisions and implementations.
Solving the versioning problem
It’s quite essential to store different user contributions on the same subject separately if the application supports crowdsourcing. We cannot let the user to overwrite the existing data without any moderation. Hence in Grandham, we have a ‘Book’ model with many ‘Submissions’ out of which one will be an approved submission. A book’s individual page basically has the detail from this approved submission.
It will be task force (a set of users with advanced privileges) who would approve submissions. They will have options to comment or even edit a submission.
Thanks to Travis CI, Grandham project now has a Continuous Integration server. We also use Gemnasium to track dependency packages.
The first week of development which started on 17th of June essentially comprises requirement analysis, planning and API implementation.
Though the basic requirements for Grandham were discussed a month before, it was necessary to have more clarity on the subject. I had a series of discussions with mentors on various aspects of application, especially on how this project would attract information from contributors of different competency. It was sorted out that normal user could provide just the basic information such as Title, Author(s), Publisher(s), Year, Pages, Edition and Description. Advanced users, who are competent with MARC21 format could contribute advanced bibliographic information through a specially designed interface.
As mentioned in the previous post, we had a meeting with Sri K. H. Hussain. Apart from inspiring and guiding us, he made us aware about the importance of having an API and integration with various library systems in existence. I especially remember seeing a live instance of Koha and its super complex MARC21 data input form. It will be really great if it could consume Grandham API and vice versa so that we would have proper data sharing between two information resources.
I had little confusions around how to store information. I was thinking of extracting basic information from complex MARC21 data to display in individual books page. But since it was decided that users feed them differently, we could easily store basic information separately and use them. Separate data models to store information as Key/Value pair were written, but I removed them all for the sake of readability and easiness. It was this period when we decided to integrate Bootswatch for front end CSS framework.
Books and Fields API were written during period using jbuilder. The data importer script was updated to meet the changes in the application.
A couple of weeks before, We (Anivar, Manoj and me) had a meetup with Sri K. H. Hussain who is a pioneer in library science. He has worked in many projects including earlier version of Grandham (Malayalagrandham.com) and has been a member of its task force. He offered guidance for development and his inspiring vision for this project helped us to fix the goals.
(The trip for the meetup was fun. Though it was for Google ‘Summer’ of Code, we got drenched in the heavy monsoon of Kerala ;))
Meanwhile the coding period for GSoC had started, It was time to fasten the development process and get the project up as fast as possible. I had drawn few design mockups and have started implementing them.
Most of these designs are just helpers to assist in development to get the backend and features done. We will do proper styling and UX enhancements after implementing the backend.
Currently the application has the following features,
- Basic books listing and API
- Bootswatch integration
- Dynamic URLs in Navbar according to the language selected
- Add book form
- Edit book (which would save the data as as a new revision)
- Decent test coverage
Live instance of the application in development is running here – grandham.herokuapp.com
Regarding the representation of binary MARC21 data in relational database, we had decided to convert it to MARCXML and then it shouldn’t be a problem to store. It needs a bit more research on how many attributes and data fields it would take to completely express MARC data, hoping to write a separate post on that later.
That’s all for now. Happy hacking, folks!
I’m very happy to announce my recent hobby project – QuizGrid.com. It is a social network for people who love quiz.
I’ve been observing an increasing number of people in facebook who make and join groups to ask GK questions for information sharing and fun. But facebook group is not built for quizzing and the whole fun of finding the answer remains until the first guy finds and posts it as a comment. QuizGrid essentially solves this problem, answering and commenting on a question are different entities in QuizGrid. Also the support for questions with options is another advantage among many other features in this quizzing platform.
Unlike the traditional quiz pages over web, QuizGrid gives you a social network experience with the feed of questions ranked according to the users you follow. QuizGrid also shows actions like ‘Bob and Alice answered this question’ or ‘John, Tom and 4 others commented on this question’, and so on if you follow Bob, Alice, John, Tom and those 4 other users. QuizGrid also has data crunching algorithms to show you relevant and concise real time notifications.
As a software developer, programming this project was immense of fun and challenging. I remember rewriting the entire code written using ORM for extracting relevant questions based on the actions of following users with raw SQL queries since the ORM implementation became very on a simulation test with thousands of questions and users. There’s definitely room for improvement and new features. Since it’s a hobby project, I develop QuizGrid during nights and weekends, I look forward to release more features and performance fixes in coming weekends.
Please sign up at QuizGrid.com if you are interested in Quiz and networking with similar minded people. Cheers!