{"id":1657,"date":"2018-06-07T17:19:10","date_gmt":"2018-06-08T00:19:10","guid":{"rendered":"http:\/\/yan.lalife.net\/?p=1657"},"modified":"2026-01-20T05:08:20","modified_gmt":"2026-01-20T05:08:20","slug":"softmax-vs-sigmoid-function","status":"publish","type":"post","link":"http:\/\/www.lalife.net\/?p=1657","title":{"rendered":"Softmax vs. Sigmoid function"},"content":{"rendered":"<p>We are familiar with Sigmoid function when learning Logestic regrestion, as well as in Neural Network as the non-linear active function.<\/p>\n<p>Softmax function is another type of non-linear function<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft wp-image-1658\" src=\"http:\/\/yan.lalife.net\/wp-content\/uploads\/2018\/06\/softmax1-300x87.png\" alt=\"\" width=\"157\" height=\"54\"><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>While Sigmoid is as the following in math,<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft wp-image-1660\" src=\"http:\/\/yan.lalife.net\/wp-content\/uploads\/2018\/06\/sigmoid-1-300x69.png\" alt=\"\" width=\"154\" height=\"45\"><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>How to separate them in usage?<a href=\"http:\/\/dataaspirant.com\/2017\/03\/07\/difference-between-softmax-function-and-sigmoid-function\/\">&nbsp;Here<\/a> is a good explanation.<\/p>\n<p>Also some good example for Softmax in <a href=\"https:\/\/en.wikipedia.org\/wiki\/Softmax_function\"><em>Wiki<\/em><\/a><\/p>\n<p><em>If we take an input of [1, 2, 3, 4, 1, 2, 3], the softmax of that is [0.024, 0.064, 0.175, 0.475, 0.024, 0.064, 0.175]. The output has most of its weight where the &#8216;4&#8217; was in the original input. This is what the function is normally used for: to highlight the largest values and suppress values which are significantly below the maximum value&#8230;<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are familiar with Sigmoid function when learning Logestic regrestion, as well as in Neural Network as the non-linear active function. Softmax function is another type of non-linear function &nbsp; &nbsp; While Sigmoid is as the following in math, &nbsp; &hellip; <a href=\"http:\/\/www.lalife.net\/?p=1657\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[],"class_list":["post-1657","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"_links":{"self":[{"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/posts\/1657","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.lalife.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1657"}],"version-history":[{"count":1,"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/posts\/1657\/revisions"}],"predecessor-version":[{"id":1833,"href":"http:\/\/www.lalife.net\/index.php?rest_route=\/wp\/v2\/posts\/1657\/revisions\/1833"}],"wp:attachment":[{"href":"http:\/\/www.lalife.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1657"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.lalife.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1657"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.lalife.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1657"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}