« first day (297 days earlier)      last day (2933 days later) » 

4:01 PM
1) I sort of followed the API of neural network toolbox: mathworks.com/help/nnet/ref/configure.html, where the configure method is used to initialize input/output sizes (in our case we already did that in constructor) as well as their ranges (this is used in case of pre/post processing)
2) no, each neuron is connected to a bias term, we can't remove it
3) yes, in the tensorflow demo, the output layer is always of size 1, hardcoded with a 'tanh' activation function in case of classification, and 'linear' function in case of regression
@rayryeng forgot to ping you
 
4:20 PM
@Amro ok so there's a bias neuron at the output layer? I've never seen that before but ok
I always keep the bias units for the other layers but never for the output when it's time to return the predictions. They are superfluous mainly because you aren't their weights to the other neurons in the output layer but OK.
Updating their weights.
 
 
1 hour later…
5:24 PM
@rayryeng um no, there is no bias on the output layer itself, rather the neurons in the output layer are connected to a bias from the previous layer
 
5:44 PM
Today I learned that Matlab has a table datatype (since 2013b). I found out via this answer, which could use more upvotes to be more visible. (I hope it's OK to ask for upvotes this way - it's a good answer, and the community would benefit from the visibility. I have no connection with the user Sh3ljohn)
 
6:01 PM
@MartinJ.H. Tables have been around for quite some time. We all use them. Thanks though.
@Amro yup thanks. That I know. Just coding this as efficiently as possible. I'm also taking a batch size into account.
 
6:15 PM
@rayryeng I'm sure you already know this, but here's a diagram of how the network would look like:
 
6:37 PM
@Amro yup I do but that does help! BTW I misread the code. When you get to the output layer there is no bias. That's what you get when you rush reading something. The forward prop is totally fine :) Thanks!
Really useful to make sure that I know how the weight matrices are sized. It's what I expected.
 
6:52 PM
@rayryeng oops, the input layer there obviously isn't connected to a bias, it's pure input... let me fix that
the code is correct, just a drawing mistake :)
 
@rayryeng: I was merely fishing for upvotes, not trying to educate anyone on tables. I noticed that the tables-related question has 55k views, which makes it the 86th most viewed Matlab-question on SO, and it bugs me that the accepted answer is obsolete. Obsolete answers are, of course, a well-known problem on SO, with many proposed solutions... But since the score of the new answer is unchanged, I now know that fishing for upvotes in chat is not a viable solution. :-)
 
@MartinJ.H., that question has a solution using dataset, which is really the predecessor of tables in MATLAB (it was ported from Statistics Toolbox to core MATLAB and renamed as table class)
the implementations of dataset and table are almost identical
 
@Amro - thanks, I did not know that the table class is essentially the same as dataset! I unfortunately don't have a license for the Statistics Toolbox, so I can't try it out...
 
of course dataset is deprecated now in favor of table, it says so at the top of the doc page: mathworks.com/help/stats/dataset-class.html
 

« first day (297 days earlier)      last day (2933 days later) »