HellOnline

Icon

Eran's blog

  • Frontpage
    Return home
  • Browse
    By topic
  • Subscribe
    RSS feed
  • Events
  • Fiction
  • gaming
  • General
  • Java
  • Projects
  • Ruby on Rails
  • Social Software
  • The Net
    • Aggregation
    • MicroFormats
    • Mobile
    • Search
    • Tagging

November 4, 2008 • 11:59 am Comments Off

AvaPeeps Update

AvaPeeps: FlirtNation has been progressing quite rapidly. In the last few months since we’ve launched on the web we’ve been hard at work, streamlining the user experience, making it easier to join and more fun to play.

Mobile and Web still offer quite different experiences and creating a game that works well on both and is fun and engaging on both platforms is an interesting challenge. The last couple of updates focused mostly on making the user experience on the web more fun, taking advantage of more screen real-estate and better UI options. It shows too, the game flows much better and we see more people enjoying it every day. We’ve also been noticed by the blogosphere. Inside Social Games, writes:

What is most interesting, however, is that AvaPeeps was originally a mobile game that migrated to the web space (rather than the other way around). That said, you are also capable of playing the game using T-Mobile, BOOST, Virgin, or Three (in the UK). Mobile games isn’t particularly unusual, but in this case, the game has been tied to the web version as well, allowing players to interact with each other in real time, from anywhere, regardless of whether or not they are using the web or a mobile device.

We’ve got plenty more planned for the next few months, so stay tuned!

Filed under: Projects , avapeeps, digital chocolate, gaming, Mobile, mososo, Social Software

March 6, 2008 • 4:31 pm 3

Twitter Killfile

Now that everyone’s gone to SXSWi and left me here, alone in the Bay Area with nothing to do and all this parking (not really, there’s still no parking) I finally have some time to be productive. To make sure that my twitter stream remains somewhat useful for the next 10 days or so I created this small Greasemonkey script to filter out all that South By Noise(tm). It might not be useful if you use twitter from some desktop client or IM but if you’re like me and you’re still using Firefox this may actually be useful.

Notes:

  • Yes, you need Greasemonkey to run this.
  • Yes, it probably has bugs.
  • Yes, it may cause your computer to implode and your entire online identity to be sucked into the ensuing vortex. Use at own risk!
  • Last, I based the script on the Reddit Content Filter script by pabs. So, thanks dude!

Oh, install the script and add banned words and authors using Tools > Greasemonkey > User Script Commands > Edit Blocked (Users/Keywords)…

Update: Slight bug fix. Latest version is 0.21

Filed under: Projects , firefox, greasemonkey, sxsw, sxswi, twitter

October 2, 2006 • 4:40 pm Comments Off

Re: Microformats and SIOC

John Breslin writes about the overlap between Microformats and the SIOC (Semantically-Interlinked Online Communities) project. SIOC was one of my references when expanding cite-rel and later when designing my Distributed Social Anything project so, naturally, I’m interested.

John brings up a few points where SIOC and microformats could help each other and therefore help make the Web more semantic.

In the SIOC and FOAF vocabularies, there are two properties linking people to user profiles: a Person is linked to a User using a “hasOnlineAccountâ€? link, and a User is linked to a Person using an “account_ofâ€? relationship.

To the best of my knowledge, the way this is normally handled in the mf world is based on XFN (XHTML Friends Network) and possibly with the addition of hCard. Using XFN rel=me links one can establish an equivalence relation between different accounts to show that they belong to the same person. Additionally, one could use a rel=me link to an hCard to provide further information about the actual person (as opposed to an account on a website). This is described to some degree on the GMPG site but that description is getting somewhat out of date and should really be moved to the mf wiki.

…when you create your post, link back to your own post aggregation resource. From the user side, this could be achieved by creating a mf for sioc:has_container that allows linking to “Virtual Forumsâ€? from the post creator side, e.g. from the post content.

I’ve faced this problem while designing the DSA system and chose a similar solution based on an existing microformat – rel-directory. By adding ‘directory’ to the rel value of a link, the author states that the destination of the link is a container listing an entry for the current page. Using the rev attribute one could publish the complimentary relation.

From forums, a mf for has_part / part_of would allow one to link a forum or blog or any discussion channel to a larger community, e.g. this could be done from a forum description. Also, a mf for has_part / part_of would help to link a user to a community, e.g. by creating a typed link from a user signature.

As John says, this is very close to the word I’ve done with DSA. I haven’t really thought of showing relationships between whole forums and groups but this can be done in the same way as single posts using rel-directory (mentioned above). For users, I wanted something with a little more specific semantics that’s easy to integrate with XFN so I’ve started work on XMF (XHTML Membership format).

Lastly, John touches on distributed conversations and cite-rel. There is, indeed, some correlation between cite-rel values and SIOC/IBIS relations but there’s also an important difference.

Ultimately, we need ways to say that a post is in agreement or disagreement with a previous post or even with specific parts of a previous post (see picture below). Also, we may need to describe other reply types that are needed, as with IBIS. (Note: rev-update and rel-update may correspond to sioc: previous_version and sioc:next_version respectively, and via may be compared with sioc:related_to.)

Ryan and I created cite-rel with the intent of keeping it simple. In the microformat way of focusing on the 80% case we mostly dealt with simple relationships between posts (replies, forwarding, updates and via links). We left dealing with the subtle nuances of agreeing/disagreeing to future specs. In this specific case, we can actually use one of the earliest microformats, vote-links, to show the authors attitude towards a previous post. As for other values, here’s the mapping as I see it:

sioc:has_reply – rel-reply
sioc:reply_of – rev-reply
sioc:next_version – rel-update
sioc:previous_version – rev-update
sioc:related_to – I’d say that via can be mapped to sioc:related_to but not necessarily the other way around as via links have a very specific meaning that is not encoded in SIOC as far as I can see. Perhaps IBIS’ source would work?
IBIS:pro/con – map very well to vote-for and vote-against in vote-links.

Filed under: Aggregation, MicroFormats, Projects, The Net

August 18, 2006 • 5:25 pm 3

HowTo: Using Acts as Wiki to Create a Smarter Blog

This post is a tutorial on how to use the Acts as Wiki rails plugin. It assumes you’re already somewhat familiar with ruby and ruby on rails. If you’re not, check out the rails website, it has some good starting points. You can see the full source code for this project on trac or download it as a zip file.

It’s common practice among bloggers to mark updates to existing posts with <INS> and <DEL> tags or similar methods. Wouldn’t it be easier if the content of each post was a wiki? Anyone would be able to see the full revision history of every one of your posts and full disclosure would be served. Using the Acts as Wiki plugin we can achieve just that.

First let’s create the database we’ll be using, call it a bliki (blog + wiki). Once that’s done go ahead and create the rails application itself

rails bliki

Edit database.yml to include the correct details and then go on to create the Posts table, we’ll use a migration for that:

class CreatePosts true
t.column :subject, :string
t.column :content, :text
t.column :created_at, :datetime, :null => false
t.column :updated_at, :datetime, :null => false
end
end

def self.down
drop_table 'Posts'
end
end

Run the migration

rake db:migrate

Now create your Post scaffold.

ruby script/generate scaffold Post

It’s now time to install the Acts as Wiki plugin, just like it says in the Readme:

script/plugin install http://www.hellonline.com/svn/hellonline/plugins/acts_as_wiki/
script/generate acts_as_wiki
rake db:migrate

Now lets use acts_as_wiki on the content field of the Post class and throw in a markdown fitler as well, just to make things pretty

class Post < ActiveRecord::Base
acts_as_wiki :content, Filters.method(:markdown)
end

Let’s display the all the posts in a list, you can see the full view code here but the interesting lines follow:

And later

'revisions', :action => 'history', :type => post.class, :o bj_id => post.id, :title => :content %>

The first one displays the post’s content after passing it through the markdown filter (if you want the raw content, just use post.content as you would otherwise). The second one links to the revision history of this field. The link requires the containing object type and id and the field’s name (supplied under :title). We need to make similar changes to show.rhtml too.

We can also make use of the new update_attirbutes(attributes, author_name) method. Go into post_controller.rb and change the update method to the following:

def update
@post = Post.find(params[:id])
if @post.update_attributes(params[:post], params[:post][:author_name]) # 'show', :id => @post
else
render :action => 'edit'
end
end

This will record the author name along with the new revision. If you want to record that detail for revert actions as well, you’ll need to change revisions_controller.rb. In the revert method change the call to obj.revert() to something like:

obj.revert(@revision.field_name, @revision.id, @session[:username])

This assumes, of course, that you’re using the session variable username to store the current user’s name.

To make things a bit clearer change the post’s _form.rhtml partial, we don’t need to provide anything beyond an author’s name, a subject and the post’s content:

Author name

Subject

Content

That’s it. Start your server and go to http://localhost:3000/posts/. Create a new post (you can use some markdown if you want to), edit it, check out the history, compare revisions, revert to a previous revision. Isn’t this fun?

Filed under: Projects, Ruby on Rails

August 16, 2006 • 5:48 pm 1

Acts as Wiki

A wiki is a very useful thing. In fact many times I’ve wanted to have just one field in a class be a wiki. This behavior is even more desirable with all the Web 2.0 user-generated content type sites coming out lately. The way I see it, Wikis have two useful features:

  1. Revision history – so you don’t have to worry too much about vandalism, opinionated authors, etc.
  2. Markdown or similar filter to make rich content authoring simple and safe.

One of my first additions to WhereAreYouCamping.com (WAYC) was the versioning code. Anybody can edit camp content and so the site is left wide open to vandalism. The code for that is pretty simple


class Camp 'Revision',
:foreign_key => 'camp_id',
:conditions => "field_name = 'description'",
:o rder => "id DESC"
before_save :revise(‘description’)

The revise function is in charge of creating a new revision if the current value for the description field is different from the previous value. I’m omitting it for brevity.

This is very simple and works pretty well as a simple backup method. You can probably imagine that with some pretty simple code in the Revisions_Controller it easy to show the revision history of a field, diff revisions and revert to a previous one. On the other hand, this solution is lacking author details and a few other niceties but the biggest disadvantage is, of course, lack of DRYness. This method is not at all reusable; in fact additional wikified fields requires much replicated code. Enter acts_as_wiki.

Acts_as_wiki is a mixin (soon to be a plugin, I hope) that lets you do all that with one line of code:

acts_as_wiki [:description, :events], Filters.method(‘markdown’)

Adds revisions (with history, diff and revert) and markdown filtering (or whichever filter you decide to use if at all) to the description field. Here’s a simplified version of the method:


def acts_as_wiki(field, filter = nil)
# remember the filter for this field.
Filter_hash[field.to_s] = filter
# add reader to filtered field content
module_eval "def #{field}_filtered self.get_filtered('#{field}'); end"
# add the revisions relation
has_many("#{field}_revisions".to_sym,
:class_name => 'Revision',
:foreign_key => 'obj_id',
:conditions => "field_name = '#{field}' and obj_type = '#{self.class}'",
:o rder => "id DESC",
:as => 'obj')
}
before_save Proc.new{|obj| obj.revise(field)}
end

To record the author who created a revision I’ve updated the update_attributes method:


attr :current_author, true

def update_attributes(attributes, author_name = 'unknown')
self.current_author = author_name
# let super handle the actual update
super(attributes)
end

Later in revise() I refer to current_author to get the author’s details. This can be expanded to support a more complex user model. For content filtering I’ve added the get_filtered method which is used by the FIELD_NAME_fieltered method (mentioned above):


def get_filtered(field_name)
content = self.send(field_name)
filter = Filter_hash[field_name.to_s]
if filter
return filter.call(content)
else
return content
end
end

This allows the programmer to continue accessing the raw field value (as stored in the database) using the field name and so requires very little change throughout the application.

Revisions are simple creatures:


create_table "revisions" do |t|
t.column "created_at", :datetime, :null => false
t.column "revised_at", :datetime, :null => false
t.column "content", :text, :default => "", :null => false
t.column "author", :string, :limit => 60
t.column "obj_type", :string
t.column "obj_id", :integer, :null => false
t.column "field_name", :string, :limit => 20
end

Similarly, the Revision model needs very little besides the relationship to its owner:


class Revision 'obj_id', :polymorphic => true
# some helper functions for browsing through revisions
def prev_rev
Revision.find(:first,
:conditions => ['id 'id DESC') rescue nil
end
def next_rev
Revision.find(:first,
:conditions => ['id > ? AND field_name = ? AND obj_type = ? AND obj_id = ?', self.id, self.field_name, self.obj_type, self.obj_id]) rescue nil
end
def last_rev
Revision.find(:first,
:conditions => ['id >= ? AND field_name = ? AND obj_type = ? AND obj_id = ?', self.id, self.field_name, self.obj_type, self.obj_id],
:o rder => 'id DESC') rescue nil
end
end

The revisions controller is also pretty simple. The basic actions: history, show and revert pretty much write themselves. The diff action is just as easy once you use the diff method from instiki.

And there you have it. The field[s] of your choice can now be a full blown wiki with revision history, markdown, the works. You can see the rest of the code on trac.

Update: I’ve put up a plugin, you can install it using

script/plugin install http://www.hellonline.com/svn/hellonline/plugins/acts_as_wiki

generate the code and migrate your db using


script/generate acts_as_wiki
rake db:migrate

Update 2: Just posted a tutorial on how to use the Acts as Wiki plugin.

Filed under: Projects, Ruby on Rails

July 24, 2006 • 6:35 pm Comments Off

Mini Project: Where Are You Camping?

I did not think I could create a useful and interesting Rails application in 24 hours. I was wrong.

Where are you camping is a simple web application designed to help you track where all your friends are camping at Burning Man this year. It is small, focused and simple but even the initial version of the application achieved its goals pretty well. I started working on this one Thursday night and by Friday night the site was already up and running under the domain I purchased earlier that day.

Features:

  • Post information about yourself.
  • Post information about your camp.
  • Join/Leave a camp.
  • See camp description and members.
  • AJAX-y filtering of camp and user lists.
  • Mark people as ‘favorites’ (added in second version)
  • Print list of ‘favorited’ users and their camps (also added in ver. 2)

Plenty of thought went into making the site as simple as possible so that more users find it accessible. I considered waving registration and simply using cookies but then opted for simple registration. Another time consuming aspect was the UI. I’m no Javascript/AJAX wizard but thanks to RoR and Scriptaculous I managed to throw in some nice UI elements like the dynamic list filtering and sorting.

If I ever needed it, this is yet another proof that Ruby and Rails save you from wasting time reinventing the wheel (or importing every library since the invention thereof) and let you spend more time concentrating on the important things.

PS. All that said, the site is still very much in alpha. If you like it, hate it or if you came across any problems, please let me know.

Filed under: Projects

January 31, 2006 • 5:49 pm 4

Updates: cite-rel, distributed social anything

Time for a couple of updates.

cite-rel

I’ve been doing some research on distributed conversations and markup in order to bring cite-rel to a draft status. You can see the markup examples I collcted, formats used in other places and the current state of the discussion all on the wiki. Please add whatever additional information or thoughts you have on the subject to the appropriate wiki page.

Distributed Social Anything

In case you missed the blog post, there’s now a whole trac wiki devoted to the project. I’ll be pursuing this project throughout this semester at USF as my Master’s Project. I’m planning to keep the project as open and transparent as possible. I’ll try to keep the wiki up-to-date and I would appreciate any feedback from interested parties. I only ask that you create an account on the wiki before modifying anything, this will allow me to keep track of who I’m talking to.

Oh, the code-name for this is Project Nirvana. Don’t ask why.

Filed under: MicroFormats, Projects

January 29, 2006 • 1:29 pm 2

Writing a Lucene Based Search Engine (pt. 4)

Part4: Adding advanced textual analysis

In a simple search engine that has no link analysis the more you do with the document text, the better, or at least that’s my opinion. I’ve started doing some experimentation with linguistic/semantic analysis of the document text in order to improve search results and search result summary quality. These experiments were contained in a library I called Linger.

Linger is a library and an XML based format that adds a semantic/linguistic layer of analysis to the indexing and search process. The results of the analysis are expressed using simple XML format so that the information may be easily stored and retrieved. The library performs sentence boundary and named entity analysis is using LingPipe from alias-i. It is designed to work as a pipeline, each step adding more semantic markup to the document, so it is simple to add more steps to the process or change existing ones without affecting the rest.

Analysis process
Semantic analysis is performed during indexing and results in an XML document. This document is tokenized using a Linger aware tokenizer that produces a stream of Lucene tokens. For the tokenization process, sentence boundaries are ignored but named entities end up being indexed twice. A named entity token is indexed once for every token contained in it and once for the complete complex token. This should result in more significant matches on tokens belonging to named entities.

The entire document is also stored in its post analysis (linger) form in the Lucene index. During search this stored form is retrieved and is used to generate the context sensitive summary. Since the semantic information is stored in XML form it is very easy to acquire it again. The new summarizer algorithm uses the sentence boundary information to create more meaningful summaries (ones that start and maybe stop at sentence boundaries).

Tokenization of the Linger format is made to resemble Lucene’s standard analysis tokens. LingerHandler is a Lucene Tokenizer and produces Lucene Tokens, thus it can be used as part of the indexing process instead of Lucene’s standard tokenizer and analyzer. LingerHandler adds two new types of tokens to match the two types of semantic markup. The End of Sentence token and the Named Entity token. As stated before, Named Entity tokens are duplicated. The duplicate token (the token with the full named entity) has a length of 0 so it can be ignored when trying to recreate the original text from tokens.

Summary
All in all, this experiment turned out pretty well. The new summaries make much more sense than the old ones (which would start in the middle of a sentence and end in the middle of the next one) and Named Entity detection works pretty well to enhance the search results. However, I did not get to do much performance testing on any of the new algorithms involved so it is hard to quantify the effect they would have on overall performance. There is definitely some more work to be done on the named entity detection (a learning algorithms would be nice) and maybe some matching enhancements to the query processor. Tokenizing and reconstructing the linger document was not as easy as I thought it would be. I’ve started considering external markup, it might make things easier.

Filed under: Projects, Search

January 24, 2006 • 4:58 pm 1

Using a DNS-like model for Distributed Conversations

One of the goals of cite-rel is to enable tracking of distributed conversation by aggregators (like technorati or memeorandum) over multiple blogs. Using a simple microformat like cite-rel to solve the problem has the advantage of a very low cost of entry. Any user can employ cite-rel and any blog software, indeed any tool that publishes HTML can support the format. The downside to that is requiring a third party – the aggregator – and the possibly large amount of work required by that aggregator. It is possible, however, to build a different solution to the problem that would not require any third party and would only require analyzing the conversations each participant is a part of.

DNS is a distributed publishing mechanism. Each DNS server is in charge of only a small subset of the entire domain system but using recursive queries each server can serve information about every public domain. Recursive queries work as a distributed search mechanism that leads your DNS server to the servers with the authority to answer the query. Your DNS server then caches the reply for a limited time so that repeated queries for the same domain would be served faster. We can employ a similar solution with blogs.

To implement such a solution we require two elements:

  1. Support for recursive queries.
  2. A search mechanism.

The search mechanism can be based in existing web technologies – pingbacks. Blog software already supports sending pingback and tracking them, this allows blogs to store references to all replying posts. Further, when posting a reply to a post on a different blog, the blog software can keep track of the original post. With these two mechanisms in place we can completely reconstruct the entire thread, even though each blog only stores links to directly connected posts.

Recursive queries for thread information will come in two different flavors.

  1. A request for an entire thread from a specific post. This type of request will be recursively redirected to the blog hosting the original post, unless this it is already there.
  2. A recursive request for all replies starting with a specific post in a thread. This request will recursively propagate down using pingback information to all blogs that published replies to this post.

Using these two simple requests any blog can give access to full threads for every post published on it. If we add a simple caching mechanism, the performance of the system should improve dramatically without using too much space.

Filed under: Aggregation, MicroFormats, Projects

January 19, 2006 • 7:38 pm 7

Distributed Social Anything

Following are generally unstructured thoughts and plans for a possible project. I’ve been thinking about something in this vain for a while but have never put those thoughts into a more permanent form so here goes. This post serves mostly as scratch paper for my ideas so feel free to skip it if you don’t like long, raw, technical posts.

For lack of a better name, I’m calling this Distributed Social Anything. The most concise description I can come up with is distributed Tribe.net. Completely distributed (and then aggregated for convenience :) .

  • All content published and owned by the users.
  • All content is accessible by any would be aggregator and formatted according to open standards (mostly microformats).
  • Based on existing tools and technologies. The main publishing tool is a blog.
  • Compatible with current tools. The requirements to participate are few, users of most blog hosting services should be able to participate.

Features and concepts:

  • Identity is defined by a URL. Currently the entities in the system are users and groups, both will have a canonical URL that contains at least XFN data. This XFN data (slightly expanded) defines the standard social network for users but also group membership.
  • Reciprocal XFN links might be required for some of the relations defined later. This is optional and left to aggregators to decide.
  • Group membership is published in XFN. This might require reciprocal links between users and group.
  • Users can publish information about themselves using XFN and hCard. User rel=”me” to link to additional shards of identity.
  • Users publish content on their blog. This content is later aggregated by groups to create a coherent group view .
  • Channels are feeds of blog posts that belong to a specific set. Channels are defined by tags or categories. Each group has at least one channel. Posts marked with that channel’s ID will be part of that group’s discussions.
  • Discussion are annotated using citeRel. Group aggregators might display those in a threaded format
  • Displaying previous versions of posts (in case of editing) with a diff view would be nice
  • XFN links should be aggregated and searchable (similar to rubhub.com). A service that offers search in the XFN space would be very nice
  • Group aggregators should also be aware of rich data (events, listings, etc.)
  • The group site might be able to highlite specific type of rich data (images, bookmarks, etc.) and/or offer access to it using feeds/API
  • We need administrative control of the group – membership, post moderation, rules, access-control, etc.
  • API for the group aggregator
  • Note: group aggregators can collect content from many sources, not just blogs (e.g. flickr, delicious)

Existing support:

  • WordPress supports feeds for categories. Also posts can belong to more than one category. Free channels!
  • RubHub does some XFN search but does not seem to be open source :(

To do:

  • Express group membership using XFN (rel=”memberof” ?)
  • Finalize citeRel.
  • Expand and improve on structured blogging.
  • A format for publishing group information.
  • Possibly replicate and improve on RubHub.
  • The group aggregator service.

Filed under: Aggregation, MicroFormats, Projects, Tagging

Next »

Pages

  • About Me

Archives

  • April 2009
  • March 2009
  • February 2009
  • December 2008
  • November 2008
  • September 2008
  • August 2008
  • March 2008
  • February 2008
  • January 2008
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006
  • December 2005
  • November 2005
  • October 2005
  • September 2005
  • August 2005
  • July 2005
  • June 2005
  • May 2005

Meta

  • Register
  • Log in
  • Valid XHTML
  • XFN
  • WordPress

Blogroll

  • Ashton
  • Assaf
  • AvaPeeps BlogNation
  • Captured
  • Crimson Canary BioTech Search Services
  • del.icio.us
  • flickr
  • Gamasutra Feature Articles
  • Google Blog
  • Google Weblog
  • I Blame Beer
  • Laughing Squid
  • Marc’s Voice
  • Mark Pincus Blog
  • My brother’s surfboard repair shop in Waikiki
  • Ongoing
  • Prof. David Wolber
  • Riding Rails
  • Ryan King
  • supr.c.ilio.us: The Blog
  • Tantek Çelik
  • Tara Hunt
  • Technorati Profile
  • The Search Guy
  • tribe.net
  • zengestrom.com
  • Frontpage
    Return home
  • Browse
    By topic
  • Subscribe
    RSS feed
  • Events
  • Fiction
  • gaming
  • General
  • Java
  • Projects
  • Ruby on Rails
  • Social Software
  • The Net
    • Aggregation
    • MicroFormats
    • Mobile
    • Search
    • Tagging

Blog at WordPress.com.

Theme: Grid Focus by Derek Punsalan.