Deep Learning diary (2015-09-01 ~ )

Survey and recent works about Deep Learning (2015-09-01 ~ )

  1. 2015-12-03
  2. 2015-09-01
  3. 2015-09-02
  4. 2015-09-10
  5. 2015-09-29
  6. 2015-10-03
    • Spatial Transformer Networks: using BP to learn transform of spatial parts in images (translation, scale, rotation) and use its normalized form in classify problem.
  7. 2015-10-04
  8. 2015-10-05
    • Some interesting notes
      • Notices for my work:
        • Pre-processing
        • Monitor the ratio between the update norm and the weight norm, This ratio should be at around 10-3. If it is much smaller then learning will probably be too slow, and if it is much larger then learning will be unstable and will probably fail.
        • Weight initialization: when the nets are deep, the many weight matrices all multiply each other, so the effect of a suboptimal scale is amplified.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s