How My Neural Net Sees Blackboards (Part 2)

Previously, I discussed training a neural net to clean up images. I’m pleased to say that, using more sophisticated techniques, I’ve since achieved much better results. My latest approach is a four layer convolutional network. Sadly, the convolution throws away the sides of the images, so we get a black margin. In any case, compare the results:

Original Image -- dirty blackboard image to be cleaned with neural net

Original Image

Blackboard image after being cleaned by 3 layer neural net.

Attempt 1: 3 fully connected layers

Blackboard image cleaned by 4 layer convolutional network.

Attempt 2: 4 convolutional layers.

Much better!

Now, all these results are based on images from the same source, Dror Bar-Natan’s Pensieve. Thus, even though I’ve used separate training and test images, they aren’t really independent.  It would be much more interesting to look at how well it does when seeing a very different image… The following is Richard Feynman’s blackboard when he died.

Feynman's blackboard at time of death.

Original image (Feynman’s blackboard)

Cleaned version of Feynman's blackboard

Cleaned version of Feynman’s blackboard

(I find Caltech’s copyrighting an image of dead persons’s blackboard and coating most of the images with “Caltech Archive”, to be in really poor taste and rude.)

The only preprocessing was scaling the image by a factor of two so that things were of similar scale.

I’m trying to write a super easy to use neural net library. We’ll see where this goes.

About these ads

Tags: ,

7 Responses to “How My Neural Net Sees Blackboards (Part 2)”

  1. John Says:

    that’s quite interesting. I’m curious about the reverse: analyzing silent films, processing to find the grain and streaks, then applying them to modern digial video. Sometimes contemporary digital images/video are too “stiff”, too sharp and clear; wheras film-based media can be interesting and engaging.

    • colah Says:

      Thanks!

      > then applying them to modern digial video.

      You shouldn’t need neural nets to acheive this sort of thing. It should be pretty easy to figure out an algorithm to create appropriate noise. :)

  2. Anonymous Says:

    I know next to nothing about neural networks, but I always wanted to learn about them, so I have a couple of questions.

    First, could you recommend any decent books about neural networks for someone with 0 knowledge about the subject and very mediocre programming skill? (I only know c++ and Pascal, and not especially well either.)

    Second, about cleaning up images. This may be a stupid and potentially a little insulting question, but what’s so impressive about that? It seems like it would be trivial to record a Photoshop action that does practically the same thing. (Maybe not, I didn’t actually try…) I fail to see what makes this example of usage so cool, but that’s almost surely because I am missing something important, so don’t take this question the wrong way.

    -George G

    • colah Says:

      Hey George! Good to see you again!

      > I know next to nothing about neural networks, but I always wanted to learn about them…

      You should! They’re super cool!

      > First, could you recommend any decent books…

      I’ve had the tremendous advantage of Michael Nielsen’s guidance in learning this, through a reading group he’s running.

      He’s presently writing a book, and I’m very confident that it will be awesome when it comes out! In the mean time, I’m told the standard books on the subject are by Haykin and by Bishop.

      > Second, about cleaning up images. This may be a stupid and potentially a little insulting question, but what’s so impressive about that? It seems like it would be trivial to record a Photoshop action that does practically the same thing. (Maybe not, I didn’t actually try…) I fail to see what makes this example of usage so cool, but that’s almost surely because I am missing something important, so don’t take this question the wrong way.

      Firstly, my primary motivation is that I want a problem to play around with neural nets with.

      But I think it’s a lot less trivial than you might think! It took me quite a bit of time to do the training examples by hand in GIMP, and I tried applying some standard computer vision techniques, to very limited success.

      I’d be very curious what you come up with by trying to clean an image in photosphop.

      Chris

      • Anonymous Says:

        After 5 minutes in PS I produced this:
        http://imgur.com/VtA0HB7

        It’s roughly as good as the output of your network. I think the network (especially the second one) is slightly better than me, because the pictures come out neater and cleaner than mine. I know that if I put a little more effort into it, I could match the quality of the network’s output, though.

        BUT! I shamelessly cheated. I used a cleaning procedure that is easy for a human, but would be non-trivial to script: I cut the image into 2-3 slices that had roughly the same level of darkness, and then thresholded each of the pieces individually. Cutting into even greater number of pieces would give an even nicer result, but I was lazy.

        I suppose it might be possible to program that action. You could make the program blur the image considerably, and then cut it across the level lines of luminosity value. But it would not be completely trivial.
        So, I can say that this exercise has given me a greater appreciation of the usefulness of your network. Plus, I realize that you were just messing around and wasn’t trying to code anything groundbreaking, anyway.

        I actually have to clean images like that fairly often. It occurred to me that if you turned your code into a tiny, easy to use program with even a rudimentary GUI and made it available for download it might actually make my life easier.

        -GG

        • colah Says:

          Neat!

          I’ll probably make this avialable, or something that willl make making things like this easy, or both, once I have things working a bit better.

  3. DRPR Says:

    Nice post. Do you provide source code for the 4 layer convolutional NN someplace (messy code is also fine)? Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 462 other followers

%d bloggers like this: