Submitted by Steve_Sizzou t3_zasxg5 in MachineLearning

Is it correct that with a sufficiently large - in terms of layers and nodes - neural network, that when trained, the network kind of performs feature engineering? I know that would not be formally how to describe it, but does a neural network find interesting patterns in the data that are kind of like features that are maybe even difficult to describe? Here's an example to describe what I'm getting at.

just say I'm trying to predict what picture a person is looking at based upon their brain activity, which I measure with EEG as a time series, across 128 electrodes. with a neural network can I just feed in the raw time series voltage recordings, and that will take case of discerning any valuable features, or should I also create a bunch of feature from the data - like mean, std. dev, median, entropy etc?

Thanks!

17

Comments

You must log in or register to comment.

Tezalion t1_iynlekp wrote

The overall tendency is that AI is better at such research than human, so more work delegated to AI is better. But if your system is not very advanced and refined yet, it could benefit from additional features.

−7

Duodanglium t1_iyntcgw wrote

The network only finds balance to match the inputs to the target, it will not create a feature.

A feature needs math and a purpose that you've provided; think of it as something more complicated than the starting features. For example, dividing two features while adding a third one.

−6

michelin_chalupa t1_iynu2rx wrote

Some of those additional features are going to be correlated, or otherwise implicit. I think the most basic thing you’d want to do is some smoothing.

Maybe even try DFT representations, depending on how your signals behave (not familiar with EEG data myself).

7

LanchestersLaw t1_iyof6it wrote

In general: i depends on a lot of things.

For your specific case: it will almost certainly be better to feed it the raw data.

There were a couple points in your post that seem like fundamental (but common) misunderstandings of neural nets.

  1. you cannot scale it up infinitely, that causes over fitting there is an optimal size, finding that size depends on the problem
  2. summary statistics are not features. Yes it sometimes makes sense to calculate features, not it almost never makes sense to give it summary statistics
  3. yes neural nets identify “features”, no they are not anything you would recognize as a feature
2

dancingnightly t1_iyolsht wrote

For a neural network, no you want to train it to represent the raw data (or near to raw like FFT) as other answers mention.

You could create a simple baseline Logistic regression model to check this. When you think about it, part of that model calculates the mean of each feature (128 electrodes * t times) as part of it's model(for binary classification). So even in this case, providing the mean isn't useful.

What would benefit from feature engineering?

If you have excel-table-style or low amount of data.

A classification Decision Tree is more likely to benefit - but this still usually only works if you can do some preprocessing with distances in embedding or other mathematical spaces, or augment the data format (e.g. appending presence of POS tags for text data, useful for logistic regression too).

A decision tree (usually) can't so easily implement things like totalling individual features, so things like totalling can on occasion be useful when you have low data numbers (although in theory, an ensemble of trees[which is the default nowadays for most] can approximate this and would if useful) - another example would be e.g. precalculating profit margin from the costs, net/gross profit for a company prediction dataset.

1

alex_lite_21 t1_iyp018p wrote

Deep NN do, just NN do not. In some cases the features found are not enough for good result. In some cases outperforms humans (CNN). Also, you can help the DNN by transforming the data.

2

Username912773 t1_iypqbeg wrote

Std dev mean median mode entropy kurtosis and skewness

1

calebgray t1_iyptvsg wrote

Give GitHub Copilot a shot and tell me whether or not AI-Assisted development can be counted as creating a feature. IMHO, It’s more of a philosophical debate over where re-inventing the wheel is occurring versus creating novel end results out of “the same old parts.”

−3

o--h--o t1_iypyp0m wrote

The purpose of the convolutional layers of a CNN is to perform feature extraction. You can think of it in terms of classical digital processing techniques where convolutional operators have been used to extract specific features from images like edges and boundaries or correlations. The convolutional layer can be thought of as a way of learning convolutional operations to extract relevant features. Likely you will have to do some feature engineering beforehand though as you may want to preprocess by normalizing or downsampling etc, but it depends.

2

A_the_kunal t1_iyqhdba wrote

Following this question, suppose we have a tabular data and we trained a Deep NN here. Do the weight of first layer signify the feature importance of individual features?

1