tag:blogger.com,1999:blog-174268342024-03-07T13:16:37.902-08:00MindRetrieve BlogMindRetrieve - an open source desktop search tool for your personal webTung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.comBlogger47125tag:blogger.com,1999:blog-17426834.post-1149110266317458352006-05-31T13:31:00.000-07:002006-05-31T14:41:41.150-07:00From Simple to Complex - A Reflection on Framework DesignFor a while I have used the Jakarta Struts framework in web development. I was quite overwhelmed by the number of configuration files and entries I have to wade through to get anything done. All these configurations are put in for a reason. The offer users a degree for flexibility. These are knobs you can tune to set the system up. With these knobs you no longer need to open the box and rewire things when you want to fine tunes things later.<br /><br />Alas this makes a steep learning curve. It takes a lot of work to get just some simple thing done. Also when things go wrong, it is very hard to track it down. Unhappy programmers are the result.<br /><br />It just occur to me that there is a pattern in this kind of issues in framework design. A lot of simple things adds up. When looked at a system scope, it results in something very complicated. The options of struts all serve a purpose and is not hard to understand by itself. But because you need to use all of them in your application, the overall pictures is complicated.<br /><br />I will use foreign language learning as an example. You learned and memorized many grammars rules: singular v.s. plural, subject v.s. object and so on. You went through the drilling exercises quiet well. Then the challenge came when you need to read or write a full sentence. Your tongue tied, mind gone blank. Why? Because a real life sentence is free formatted. Out of the many grammar rules you've learned perhaps two or three is relevant in the sentence in front of you. The hard part is you don't know which two or three should apply. Although you understand each rule individually, together they become a challenge.<br /><br />Some lessons I have learned as a user and designer of framework are:<br /><br />1. Look at things at system level, not just individual feature level.<br /><br />2. Use common pattern that user have already learned. Clever one-off trick may be an advantage on individual level, but it complicates the system as a whole.<br /><br />3. Design a framework can do simple work without much training. User can learn gradually when they need to do something complex.<br /><br />4. Spend time to learn the detail of the system. Like the foreign language analog, sometimes there is no shortcut but to learn really hard. If you are a user who stuck with a complex system you have to use this applies to you. For framework designer, better stuck with point 3 than assume your users' dedication.<br /><br />5. Finally, provide a tool to manage complexity. If an action have to involve 2 beans, 3 templates, and 5 validation rules, make a tool to help user to track them down. Surprising much of my time in software development is spent simply in tracking these things down manually.<br /><br />Rudy on Rail's convention over configuration philosophy is a good example. It uses a simple rule consistently (naming notation as the main logic). It can get simple thing to work easily and complex thing possible.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com2tag:blogger.com,1999:blog-17426834.post-1148970273812044612006-05-29T23:13:00.000-07:002006-05-29T23:24:33.823-07:00MindRetrieve got AjaxThanks for the <a href="http://prototype.conio.net/">Prototype</a> library, MindRetrieve has started to add some Ajax UI, like an auto-completer for entering tags makes it so much easier to from pick existing tags.<br /><br />I have spent much time struggling with Javascript and the poorly documented DOM before. Relying on trial and error and tribal knowledge to program DHTML is so painful. I'm glad to find these new generation Ajax libraries has brought a big improvement to this situation. They have done things the DOM standard has failed to do, namely to provide a consistent interface to programming dynamic elements across browsers.<br /><br />Prototype's implementation has a lot of neat things. The $() method as a shorthand to document.getElementById() is just brilliant.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1148401458862791992006-05-23T09:10:00.000-07:002006-05-23T09:24:18.876-07:00Took a RSS breakI have took a 2 weeks break in reading the technology blogs and industrial news. Now each of the 20 feeds I've subscribed has at least 50 unread items. Not knowing the 20 new products that has since launched, I feel that I am out of touch with the industry.<br /><br />It is interesting how much a RSS feed can change a person. As long as I am keeping up the news I feel like I am an active player. But the world moves so fast that if you standstill for a little while you will have a lot of catch up to do.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1146152148896068612006-04-27T08:26:00.000-07:002006-04-27T08:35:48.910-07:00berlios outageberlios, the hosting service of this project, is suffering from another outage. This time I'm counting 16 hours since yestarday afternoon. Over the course I've seen many small outages that last minutes or a few hours. I should get very serious about finding alternative provider, especially now that sourceforge offer subversion finally.<br /><br />For now, I hope berlios will get back to life soon.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1144710910814142632006-04-10T16:08:00.000-07:002006-04-10T16:33:41.143-07:00Fixing TortoiseCVS's shattering glass sound<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://photos1.blogger.com/blogger/6899/1679/1600/TortoiseCVSSound.jpg"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;" src="http://photos1.blogger.com/blogger/6899/1679/320/TortoiseCVSSound.jpg" border="0" alt="" /></a>TortoiseCVS plays an irritating sound when it encounters an error. It is downright embarrassing when you are sitting in a cafe and from your laptop comes this <br /><a href="http://cvs.sourceforge.net/viewcvs.py/*checkout*/tortoisecvs/TortoiseCVS/build/TortoiseCVSError.wav?rev=1.2">shattering glass noise</a>.<br /><br /><p>What's annoying is I couldn't find a way to change it or turn it off. I tried the usual places and searched the web. Still I couldn't get the noise to leave me alone. Finally I found the right place to do it. Go to Windows - Control Panel - Sounds and Audio Devices - Sounds Tab. Change the TortoiseCVS Error sound to something more gentle.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com10tag:blogger.com,1999:blog-17426834.post-1144214126677553192006-04-04T21:38:00.000-07:002006-04-04T22:15:26.706-07:00Upgrade notification with AtomOne of the biggest advantage of building a web application over a desktop one is the ease of upgrade and bug fixes. For MindRetrieve I've been looking for a mechanism to notify user for upgrade. At the same time I have to keep the mechanism non-intrusive and respect the user's privacy.<br /><br />After looking at different mechanisms like CGI and Javascript on browser, it just dawn on me than upgrade notification has a lot in common with news syndication. All I have to do is to publish a news feed and then put a mini-newsreader into the application. Of course the user can always opt out or even choose to fetch it using their own newsreader.<br /><br />It is a good day to find a satisfactory solution to a nuisance issue.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com1tag:blogger.com,1999:blog-17426834.post-1143532113891516212006-03-27T23:41:00.000-08:002006-03-27T23:51:47.140-08:00xfolk - MicroformatsI was looking for a bookmark sharing standard. Sure enough there is a microformat <a href="http://microformats.org/wiki/xfolk">xFolk</a> proposed.<br /><br /><blockquote>xFolk is a simple and open format for publishing collections of bookmarks. It better enables services for improving user experience and sharing data in web-based bookmarking software. xFolk may be embedded in (X)HTML, Atom, RSS, and arbitrary XML. It is one of several open microformat standards.</blockquote><br /><br />Microformat is simple conventions for embedding semantics in HTML to enable decentralized development. It remixing of websites possible, as shown in the wonderful <a href="http://spaces.msn.com/rayozzie/blog/cns!FB3017FBB9B2E142!285.entry">Live clipboard</a> demo.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1143217957736205442006-03-24T08:22:00.000-08:002006-03-24T08:32:37.746-08:00MindRetrieve postcardThis is a postcard I've made as a promotion material. I find it a more creative compare to handing out name card. You can print it on photo paper at home. Or you can send it to a photo lab to print a big stack. Either case it probably turns out quicker and cheaper than name cards. The color is great. The 4x6 inch space afford a great deal of creativity.<br /><br />The amateurish graphics design is mine. You think I got a lot of nerve to actually hand this out to people. At least I can claim some novelty :)<br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://photos1.blogger.com/blogger/6899/1679/1600/postcard.0.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;border: thin solid #777;" src="http://photos1.blogger.com/blogger/6899/1679/400/postcard.jpg" border="0" alt="" /></a><div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com2tag:blogger.com,1999:blog-17426834.post-1143043313084896282006-03-22T07:57:00.000-08:002006-03-22T08:01:53.176-08:00Announce: interactive javascript consoleI'm rather productive in cranking out little tools lately. Here is another one.<br /><br />js_console is an javascript console you can insert into you web page. It allows you to test javascript statements interactively in the context of your web page. Try a demo and download from<br /><br /> <a href="http://tungwaiyip.info/software/js_console.html">http://tungwaiyip.info/software/js_console.html</a><div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1141852420957432522006-03-08T13:00:00.000-08:002006-03-08T13:13:40.966-08:00Google Desktop Search cannot find files movedThe story about <a href="http://www.mercurynews.com/mld/mercurynews/business/technology/personal_technology/14028721.htm">Google Desktop Search cannot find files moved</a> stir up some nosie in the blogosphere. Admittedly it is a hard problem. And it would require good platform support to do right. It is the same dilemma I have faced when implementing the tag the files feature. I choose to leave this problem alone for now (while leaving enough nexus so that it can be fixed later). This allow me to put out a simple features in days. If it is proven popular I can alway go back and spend time to resolve it.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1139854611647039842006-02-13T09:12:00.000-08:002006-02-13T10:31:26.600-08:00Announce: HTMLTestRunner - generates HTML test report for unittestI'd like to annouce the release of Python library HTMLTestRunner, a by-product of the MindRetrieve project.<br /><br />------------------------------------------------------------------------<br />HTMLTestRunner is an extension to the Python standard library's unittest <br />module. It generates easy to use HTML test reports. See a sample report at <br /><a href="http://tungwaiyip.info/software/sample_test_report.html">http://tungwaiyip.info/software/sample_test_report.html</a>. <br /><br />Check more information and download from <br /><a href="http://tungwaiyip.info/software/#htmltestrunner">http://tungwaiyip.info/software/#htmltestrunner</a><br /><br />------------------------------------------------------------------------<br /><br />Actually this was posted on comp.lang.python newsgroup a while ago. Thank you Cameron Laird for picking it up in <a href="http://groups.google.com/group/comp.lang.python/browse_frm/thread/b72778e86f40c3eb/53ae18720551697d?hl=en#53ae18720551697d">Dr. Dobb's Python-URL</a>!<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com83tag:blogger.com,1999:blog-17426834.post-1137790350492097132006-01-20T12:29:00.000-08:002006-01-20T12:52:30.503-08:00Great gathering last nightThanks to <a href="http://fuzzyblog.com/">Scott</a> of <a href="http://ookles.com/">Ookles</a> for organizing the <a href="http://fuzzyblog.com/archives/2006/01/20/who-came/">great gathering</a> in San Francisco last night. It was a great pleasure to meet so many prominent people who are pushing the Internet front.<br /><br />Geeks are officially cool again!<br /><br />Also I got to thank Scott for being the inspiration behind the new <a href="https://developer.berlios.de/project/shownotes.php?release_id=8737">tag the file</a> feature!<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com1tag:blogger.com,1999:blog-17426834.post-1137701018512083012006-01-19T12:01:00.000-08:002006-01-19T12:03:38.523-08:00Announce release of version 0.8.0I'm gload to announce the first major release in almost a year. New features in version 0.8.0 include:<br /><br />Web library - tag base bookmarking system<br />Tag based categorization<br />Tag files in local disk (Windows XP)<br /><br />Download from via the <a href="http://www.mindretrieve.net/">MindRetrieve</a> website.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1137605125531352502006-01-18T09:22:00.000-08:002006-01-18T09:25:25.540-08:00Added IE supportAfter months of developing on Firefox and Opera only, I have finally ported the DHTML code to IE. Well almost except some small bugs. IE6 is just hideous. I should really have wait to support IE7 only.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1137206386298701472006-01-13T18:32:00.000-08:002006-01-13T18:39:46.300-08:00Efficient character escapes decodingI ran into this technical issue using Python's 'unicode_escape' codecs on unicode string. Put a question in the comp.lang.python newsgroup and thought this is probably too technical for anyone to care. But the Python community never fails me. Thank you for Steven Bethard to come up with a great suggestion that works for me. I have written up the solution in a <a href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466293">Python cookbook recipe</a>.<br /><br />Hope it will be useful for other people too.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1137098305814018552006-01-12T12:15:00.000-08:002006-01-12T12:38:25.826-08:00Synchronization with Simple Sharing ExtensionsI have come across this <a href="http://msdn.microsoft.com/xml/rss/sse/">Simple Sharing Extensions for RSS and OPML (SSE)</a> specification from Microsoft. It is a minimum extensions necessary to enable loosely-cooperating apps to use RSS as the basis for item sharing – that is, the bi-directional, asynchronous replication of new and changed items amongst two or more cross-subscribed feeds.<br /><br />This seems to be a great fit for synchronizing the weblib among different repositories. The essence of SSE is a set of items, each has a globally unique id, a timestamp and an increasing version number. This has inspired me to make another revamp to the <a href="http://mindretrieve.blogspot.com/2005/12/weblib-file-specification.html">Weblib file specification</a> to add all those elements. The only remaining issue for me is the globally unique id. I need to figure out a simple way to generate them across distributed systems, (I feel UUID too heavy weight). The implementation is available in SVN, although I haven't updated the specification yet.<br /><br />Synchronization is a though issue I haven't have it all thought through. But it probably can't go wrong to go with Ray Ozzie who has created Lotus Notes.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1136996246824554102006-01-11T08:16:00.000-08:002006-01-11T08:17:26.833-08:00Quick key and the Petname systemI have come across an interesting article <a href="http://www.skyhunter.com/marcs/petnames/IntroPetNames.html">An Introduction to Petname Systems</a>. It echoes with a quick key feature I am building, which associates a short phrase with frequently used URL. For example phrase 'amex' can be used as the key to the 'American Express' web site. The main goal of the article is in security (against phishing). While for MindRetrieve the goal is to make a slick user interface. Nevertheless it helps me to understand what I am doing, an implicit system for user to assign petname to items!<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1136350971361421222006-01-03T21:02:00.000-08:002006-01-03T21:02:51.376-08:00Text v.s. multimedia?<p>The heat is on once again in the advent of internet technologies. The word is that we will move beyond text based communication to audios and videos, from text blogging to audio podcasting to video blogging, from SMS evolves MMS, from search for text to search for audios and videos, etc. The humble text is only the first step in technology development. Eventually technologies would be advanced enough to enable the full glory of multimedia.<br /><br /><p>I feel rather lukewarm for this. I often past up the news video for text article as I grow impatient with video's pace. With text I can scan back and fro much more easily. User generated content? That reminds me of those home videos, where the camera never stop panning left and right. And when the subject talks, he is actually off-screen ;)<br /><br /><p>Multimedia was the buzz world of last generation. With the enabling technology CD-ROM become popular, people envision our PC would be filled with sight and sound. While we now see lot more graphics on our PC than the early days, enough to say the focus is still on text, whether we are using email or the web. Anyway none of the multimedia companies has become Google.<br /><br /><p>I think people consider who video and audio are more advance than text are only looking at it from computer engineering's perspective. This really does not do justice to text. From a different perspective I would say audio and video more primitive because they are base on our biological sensory perception. Whether as <a href="http://en.wikipedia.org/wiki/Writing_system">text</a> is truly the great innovation than revolutionized communication.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1135960511027404512005-12-30T08:04:00.000-08:002005-12-30T08:35:11.036-08:00Incremental DevelopmentI was looking at the subversion checkin log at<br /><br />http://svn.berlios.de/wsvn/mindretrieve/trunk/?op=log&rev=0&sc=1&isdir=1<br /><br />I notice I have made many many small checkins. Often several times a day and each checkin include a group of several files (taking into consideration I am not working on this full time). This style is quite different from my work on other projects when I do a lot less checkin but usually in a larger chunk.<br /><br />Perhaps this say something about the productivity? Perhaps I was acting thoughtless because I'm the only developer right now. But just now I have come to another characterization - this is incremental development!<br /><br />Each time I made small changes, add some new feature or a methods, refactor code, fix a bug, add a test. I made the code changes, test it, and then I check in. The code base is functional most of time. Seldom did I make big changes that break the code base for several days or more.<br /><br />Is incremental development the best development process? I'll leave it to other discussion. But from a developer's perspective, having a functional system most of time and being able to test and verify any code change easily is wonderful. Everytime I do a checkin I have the satisfaction that something is done. Coming from an environment where changing only one line of code would lead to tedious work of building a test environment and a painful testing process, this is just pure joy.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com2tag:blogger.com,1999:blog-17426834.post-1135820885013380872005-12-28T17:08:00.000-08:002005-12-28T17:48:05.033-08:00Keep Your Article in One Single Web PageThere is a style widely used in web publishing. It says people don't like to read long article, one should keep modem user in consideration that a long web page would result in slow download and so. If you have a long article to publish, break it down into several short sections and let user read it page by page.<br /><br />I have arrived in a contrarian view. I think breaking a long article down into several pages is a hassle to the users. It is best to publish the entire article in one single page. The issue is in order to finish the article I need to click next page several times, every time there is a delay and it interrupts the momentum. Usually the delay is short, like one or two seconds. But it is a noticeable delay. Scrolling is seamless in comparison. For slow sites the delay is much worst. Some sites routinely take 10 seconds or more. That would feel like an episode ended in a cliffhanger and we have to wait for the next episode.<br /><br />Is several short page a better layout than a long one? I actually prefer to have a long one. The scroll bar give a good indication of how far in the article I have progressed. The scroll wheel is very handy in navigation. I can also use the browser to search for a word that appears anywhere in the article. All these is better than arbitrary break an article into several parts and then leave only a narrow window to the user. The speed issue is a non-issue. Any browser should be able to render incrementally so that you can start reading as soon as any text arrives. Even my cellphone do this flawlessly.<br /><br />In some case the multiple pages format backfire badly. I'm glad that Yahoo has a mobile version made for cellphone at http://mobile.yahoo.com/. I thought it would be great for reading email. Turns out the issue with cellphone data network is not just low bandwidth, every time when I click a link there would be a long delay before I can get any response. As Yahoo mobile break every screen into 15 lines or so, even the simplest email cost me multiple clicks to read though. The long delay between clicks make it plain unusable. I end up went back to the regular web interface. Even I have to scroll through tons of irrelevant stuff to get to the email body I still prefer it to the mobile version.<br /><br />Of course my comment should not be generalized too far. I have tried to load an entire technical manual as a single web page (in my harddrive). The browser have noticeably delay to process a web page of this size. But I still keep the manual in this format for the ease of searching.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1134752531090263732005-12-16T08:53:00.000-08:002005-12-16T09:06:39.846-08:00ISO 8601, the metric of date formatDifferent countries have different convention of writing date. Some write in m/d/y order and some write in d/m/y order and so on. One thing to do for localization is to show date in the right format...<br /><br />I say ____ the convention. Write it in ISO 8601 format. That is in the YYYY-MM-DD format. No more confusion of whether the day or month come first. And when you sort it, it comes out in chronological order. ISO 8601 is the metric of date format.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com2tag:blogger.com,1999:blog-17426834.post-1134245808744549042005-12-10T12:05:00.000-08:002005-12-10T12:19:22.086-08:00Your Site Has VulnerabilityI was testing my web application for security problem. Failure to escape user input is a very common class of security problem. So I created an input string like this:<br /><br /><tt>'"></script><h1><font size=7 color=red>GOTCHA<iframe src=http://mindretrieve.blogspot.com/2005/12/your-site-has-vulnerability.html width=500 height=300><br /></tt><br />Cut and paste it into any input fields and then click submit. If you see something you don't expect, that site probably has a problem. <br /><br />Hurry! Test out your site before hackers do!<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1134059092556901042005-12-08T08:01:00.000-08:002005-12-08T08:24:52.603-08:0017000 lines of codeIt has been a while since I did a tally on the code size. The new statistics shows there are about 17000 lines of code in about 100 modules. That is for a non-trivial application with everything deliberately designed to be as simple as possible.<br /><br />A closer look inside, among the them is 5000 lines of unit test code. It on track with the the rule of thumb of 1 to 1 ratio of production code to test code.<br /><br />Two of the largest module has around 700 lines of code. The median is around 100 lines. This sounds minuscule to most other software code. But with Python, 700 lines can be a really sophisticated module. In many cases I would break a module down into smaller components before they even reach 700 lines.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0tag:blogger.com,1999:blog-17426834.post-1134008606258582852005-12-07T18:02:00.000-08:002005-12-07T18:23:27.340-08:00Google v.s. bookmark?While we are working hard on MindRetrieve to improve bookmarking, some people argue that the entire idea of bookmarking is outdated. Instead of bookmark, we just google. Indeed Google gives such excellent experience that people expect the right answer instantaneously. More easy than you would looking things up from the bookmark menu.<br /><br />While this is true for some obvious sites, perhaps for 'dell' or 'walmart', it ain't necessary useful for everything you'd ever interested. How many times have you flip through pages of marginally useful search result until you finally found the one that gives what you need? (assuming, of course, you an advance searcher who actually go beyond the first page). You will not want to repeat this search. And if you do, you won't necessary found the same item. How about a page that is not a direct search result but is one click, or two clicks away? Let's say from Google you found a passionated travelogue about an Italy trip. From there you found more links about some hotel recommendation that you really want. Google is a good starting point. But you have also make additional effort to arrive in what you needed.<br /><br />When you search and evaluate the result, you are actually creating something of value. If you put together the link of the travelogue, a guide on how to use the Italy train system and the two cute hotel you have seen, you will in effect created some kind of travel guide, even if you have not authored any of them. Personal web is a way for you to capture the creation.<div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com1tag:blogger.com,1999:blog-17426834.post-1133455571229622862005-12-01T08:34:00.000-08:002005-12-01T08:46:11.243-08:00Weblib file specificationThe work on MindRetrieve has reached another mile stone as I have published the weblib file specification and that it supports updating now. Previously I was using a hack to rewrite the entire file whenever I update a tiny little piece of data. That actually served me well for several months during development and personal use. Rewriting a 200k file everytime never seems to add any drag. In anycase the newer format seems well designed and is more ready for future scalability.<br /><br />I've included a snapshot of the spec here or you can find the latest version from the <br /><a href="http://svn.berlios.de/viewcvs/mindretrieve/trunk/minds/weblib/store.py?view=markup">source code</a>.<br /><br /><hr /><br /><pre><br />MindRetrieve Weblib Data File Specification Version 0.5<br /><br />MindRetrieve weblib data is an UTF-8 encoded text file (no other<br />encoding is supported as this time). The overall format is a block of<br />headers followed by a blank line and then the body similar to email and<br />HTTP messages. Each line of the body part represents a webpage or tag<br />item. Update to the weblib is appended as change records to the end of<br />the file. The entire weblib can be represented by a single file.<br /><br />file = headers BR body<br />headers = *(header BR)<br />header = field-name ":" [ field-value ]<br />field-name = token<br />field-value = DSV encoded value<br />body = column-header BR *((data-line | comment-line | *SP) BR)<br />column-header = column-name *( "|" column-name)<br />column-name = token<br />comment-line = "#" any string<br />data-line = [change-prefix] (data-record | header)<br />change-prefix = '[' YYYY-MM-DD SP HH:MM:SS ']' SP ['r' | 'u' | 'h'] '!'<br />data-record = ["@"] id *( "|" field-value)<br />BR = CR | LF | CR LF<br />SP = space characters<br /><br /><br />Note<br /><br />* token is defined according to RFC 2616 Section 2.2.<br /><br />* DSV encoded value is an unicode string with the characters "\", "|",<br /> CR and LF encoded as "\\", "\|", "\r" and "\n" respectively.<br /><br />* There are two kind of data records, a webpage has a numeric id, while<br /> a tag has a numeric id prefixed by "@".<br /><br />//* A record with the same id can appears multiple times in the data file.<br />// The last record overwritten preceding records.<br /><br />* A data-record preceded by a change-prefix denote update to the file.<br /><br />* A record prefixed by "[ISO8601 time] r!" is a remove record. The item<br /> with the corresponding id is to be removed.<br /><br />* A record prefixed by "[ISO8601 time] u!" is an update record. The item<br /> with the corresponding id is to be replaced.<br /><br />* A record with by "[ISO8601 time] h!" is an header update record. The<br /> header value is to be updated. There is no remove header record. A<br /> header value can be set to empty string however.<br /><br />* The last line should always ended with BR. If the last line is not<br /> terminated with BR it is considered a corrupted record and must be<br /> discarded. Moreover never append change record to a corrupted record<br /> because the line break would be misplaced.<br /><br />* The encoding header is defined for future extension only. Only UTF-8<br /> encoding is supported right now.<br /></pre><div class="blogger-post-footer">Download MindRetrieve from http://www.mindretrieve.net</div>Tung Wai Yiphttp://www.blogger.com/profile/01409046343777567728noreply@blogger.com0