Comments for Eigenjoy

Comment on Desirable Properties for a Web Crawler by Nate Murray

Nate Murray — Wed, 14 Sep 2011 21:12:34 +0000

We’re still working on it. I recently gave a talk on how to overcome some of the challenges we faced. You can view the slides here: http://www.xcombinator.com/2011/09/14/challenges-in-large-scale-web-crawling/

Comment on World’s Fastest Binary Search? by Matt Pulver

Matt Pulver — Fri, 09 Sep 2011 17:23:40 +0000

@Chao: Thanks for that. There's also a condition in which the array index doesn't need to be checked to be in bounds every iteration. Other improvements have been made here: Binary Search Revisited.

Comment on Binary Search Revisited by World’s Fastest Binary Search?

World’s Fastest Binary Search? — Fri, 09 Sep 2011 17:16:57 +0000

[...] (Edited 9 Sept 2011: A few minor improvements have been made; see the updated post instead: Binary Search Revisited.) [...]

Comment on ActiveRecord from_xml (and from_json) part 2 by ActiveRecord from_json and from_xml

ActiveRecord from_json and from_xml — Fri, 02 Sep 2011 19:43:35 +0000

[...] (Edited 2 Sept 2011: Please see instead the updated post here: ActiveRecord from_xml (and from_json) part 2.) [...]

Comment on Desirable Properties for a Web Crawler by Daniel Ellison

Daniel Ellison — Thu, 01 Sep 2011 17:12:31 +0000

What became of this effort? I can find no further articles on the subject.

Comment on hector.rb: the pleasant JRuby Cassandra client (wraps Hector) by Sal

Sal — Sun, 28 Aug 2011 06:19:08 +0000

Great post, Nate. I look forward to playing with this. By the way, you mentioned integrating with cassandra_object, and I was wondering if you had looked at active_column (https://github.com/carbonfive/active_column). I think cassandra_object was one of the first active model abstractions (if not the first), but it doesn’t seem to be maintained anymore.

http://stackoverflow.com/questions/6157760/rails-orm-for-cassandra/6194084#6194084

Comment on Mac OS X color showing ESC[whatever for git-diff colors (and more) by Jon Erdman

Jon Erdman — Thu, 25 Aug 2011 06:17:32 +0000

Thanks, fixed me on Ubuntu/Debian.

Comment on ActiveRecord from_xml (and from_json) part 2 by Tyler Collier

Tyler Collier — Fri, 19 Aug 2011 04:19:37 +0000

In case it helps anyone else, I figured out how to populate the id’s. In the “self.from_hash(hash)” method, replace the line “new h” with:

inflated_object = new h
if h.key?(’id’)
inflated_object.id = h['id']
end
inflated_object

It makes me kind of nervous to be messing with base behavior like this…

Comment on ActiveRecord from_xml (and from_json) part 2 by Tyler Collier

Tyler Collier — Fri, 19 Aug 2011 00:39:54 +0000

Matt,

When I use from_xml, I’m seeing that my objects are inflated correctly, which the exception of their id, which stays nil. Any ideas?

Comment on Mac OS X color showing ESC[whatever for git-diff colors (and more) by not-just-yeti

not-just-yeti — Sat, 13 Aug 2011 13:25:25 +0000

Thanks, git diff and git show now work like a charm for me; in fact, they are positively beautiful in color. (I'd previously been kludging by using git diff --no-color ....) Reading less's man page also led me to its options --no-init [leave the file on-screen after finishing, rather than restore the terminal window to its pre-less state] and --quit-if-one-screen, both helpful in non-git contexts. (However, --quit-if-one-screen without --no-init is truly scary: it makes me think short files are entirely empty, since it immediately over-draws what it just displayed!) Oddly: If the environment variable $LESS is entirely unset, the defaults work well (as @Lantrix suggests). However, setting $LESS to anything at all (even the empty string) suddenly yields all-options-off rather than default-options. (I'm using OS X 10.6..8, and less --version reports "less 418".)

Comment on World’s Fastest Binary Search? by Chao Xu

Chao Xu — Sat, 13 Aug 2011 05:47:33 +0000

This is great. It is only faster than the normal binary search with a bit more optimization:
unsigned b = (1<<(31-__builtin_clz(list_size-1)));
without this line, it is horribly inefficient(at least on small arrays)

Comment on Mac OS X color showing ESC[whatever for git-diff colors (and more) by Ricardo Parada

Ricardo Parada — Fri, 12 Aug 2011 20:27:47 +0000

Thank you so much. For months I struggled to get colors to display correctly in Terminal when using git. I looked in all the wrong places and tried all kinds of things. And your post helped me fix this problem.

Comment on Chaining :include’s in Rails to reduce the number of SQL queries by Dino Reic

Dino Reic — Mon, 08 Aug 2011 08:25:33 +0000

great, thx for the effort

Comment on Mac OS X color showing ESC[whatever for git-diff colors (and more) by Brian Cunnie

Brian Cunnie — Sun, 07 Aug 2011 17:47:33 +0000

Thanks, I was having the same problem under FreeBSD.

Comment on Why is XOR the default way to combine hashes by Vijay Kandy

Vijay Kandy — Sun, 31 Jul 2011 04:24:37 +0000

Hi Nate,

Google brought me here, because I didn’t understand the use of XOR in Ryan Moulton’s article. I understand now, thanks for the explanation!

I have a question regarding calculating the final hash. Did you try (weighted or not) adding/subtracting of bits of hash values of features as suggested in [1] instead of XORing hashes? If so, can you share your thoughts comparing both approaches? Ryan Moulton’s article mentions [1] in Further Reading section.

The reason I ask is that in my simple tests, I find XORing to combine hashes of features doesn’t result in smaller Hamming distances for similar documents.

[1] http://infolab.stanford.edu/~manku/papers/07www-duplicates.pdf

Thanks and Regards,
Vijay Kandy

Comment on ActiveRecord from_xml (and from_json) part 2 by Tyler Collier

Tyler Collier — Wed, 20 Jul 2011 00:47:33 +0000

Note: In case it helps anyone else, be sure to use [Your class name].from_xml, and not ActiveRecord::Base.from_xml. Took me a few minutes to figure that out. Was getting the error: “NoMethodError: undefined method `abstract_class?’ for Object:Class”. This was obvious from the example in the older post, but would have been nice to have here too. Matt, for people like me, it’d be great if you could update the old post with a link to this one.

Thanks for the great code!

Comment on A Paging UIScrollView in Cocos2d (with previews) by Andrea

Andrea — Sun, 26 Jun 2011 16:50:18 +0000

Hi,
I have the same problem, a button outside the scrolling layer which can only be pressed when the layer is at the first or the last position. Federico have you managed to fix it?

Comment on Mac OS X color showing ESC[whatever for git-diff colors (and more) by Carlo

Carlo — Mon, 30 May 2011 05:32:11 +0000

Thanks, that did the trick on our Linux HPC.

Comment on a crawler using wget and xargs by Jake

Jake — Thu, 26 May 2011 12:42:40 +0000

Xarg & Wget stops working after about an hour

I am running your script with Cygwin on Windows XP with Dual Core and 4GB Ram
I run the script with ./crawl.sh urlfile.txt 50
I am trying to trawl through 4GB of URL

The Bash shell seems to stop after about an hour and the downloads stop even though its only 2% through the URL list

Any ideas at what could be wrong?

Comment on a crawler using wget and xargs by Jake

Jake — Wed, 25 May 2011 13:46:37 +0000

Great Work Nate

I ran your script but Xarg & Wget stops working after about an hour

I am running your script with Cygwin on Windows XP with Dual Core and 4GB Ram

I run the script with ./crawl.sh urlfile.txt 50
I am trying to trawl through 4GB of URL

The Bash shell seems to stop after about an hour and the downloads stop even though its only 2% through the URL list

Any ideas at what could be wrong?

Many Thanks