Neural networks are basically groups of containers for series of mathematical operations, both within the container and across layers of containers. "Symbolic" is referring to the fact that early researchers decided to call these containers "neurons", to symbolize the way biological neurons in our brains share "information" with each other (i.e. a network of neurons). ANNs are static in the sense that once you define the architecture (# of layers, neurons per layer, layer-to-layer connections) this architecture does not change. Your brain, however, does transform itself as you age and learn new things, hence it is dynamic.