New neural networks coming to PA

I stumbled across this a few weeks ago.
I'm coming from a more philosophical view here. Usually people having a hard time to get hold of concepts that involve more then three dimensions. So the (philosophical) question is if you wire up a human brain from the start to a 4d virtual reality simulation, would this brain be able to grasp the meaning of 4 dimensions, or is there something that stops us - in a more fundamental way - to get hold of any higher dimensions.

Within computational complexity theory there isn't any meaningful definition for dimensionality. In fact most of the problems are completely independent of the dimension of space they are embedded in. Examples are 3SAT, sorting problems, and most of the graph problems.
However we know problems like motion planning, fluid simulation, ray tracing and others that depend on the dimension of space, these are more "physical" problems and most often these are the problems we solve on an daily basis. Usually those problems behave nicely on low dimensions. For example motion planing with a holonomic drive on a plane is basically deterministic (which means if there is a solution path, you will find it in finite time). But going to higher dimensions, those problems become real beasts. An motion problem with 12 degrees of freedom can usually only solved with probabilistic methods (which means if there is a solution path, you may find it in a finite time, but there is no guarantee for it). Empirically those problems have a O-notation of O(X^d), where X is an arbitrary term given by the problem and d is the number of dimensions.

There are other measures for complexity in computational complexity than the O-notation of runtime and memory, one a bit more physical inspired is the circuit complexity (basically you build a circuit to solve a given problem, the numbers of this circuit (its depth, its width and number of gates) gives you a meaning of how difficult the problem is, like the O-notation). We know from graph theory that starting with 3 dimension you can embed any given graph, so for any problem the solving circuit can be embedded in 3 dimensions. Here it starts to get fuzzy, because there are no rules how to go from a abstract circuit graph to a "real" implementation. All complexity measures ignore things like asymptotic connectivity in space or the speed of light. The question here is how good can we embed a given circuit in a given space, when using a more realistic model.
On the other side, physics gives us a very clear framework for information and entropy, there is an upper limit of how many information a given volume in space may contain, before vanishing in a black hole. This is the Bekenstein bound, this indicates that there is a upper computational limit of our space, even if we assume our connections within a circuit do not take any space at all, the information that is carried by the circuit can be measured and gives us a upper bound on the circuit density.
So the bit more informed question is: Is there a number of dimensions d, where a class of problems with a complexity notation of O(X^d) grows so fast that it is impossible to build a circuit of the same complexity, that embeds in 3 dimension without breaking the Bekenstein bound?

Oh well, that was a lot of text.

Slightly OT but I still feel its a good place for a math joke:

The fourth dimension is space and time? And everything above that is just soul science spiritual stuff.

"An motion problem with 12 degrees of freedom", i.e. two particles moving in 3D space is a 12 dimensional problem. Each particle has coordinates x, y, z as well as momentum p_x, p_y, p_z. If you were to plot the time evolution of that system in a phaseplot, you would need twelve dimensions. It's just dimension in the sense of a degree of freedom. Or whatever structure you want to visualize that depends on multiple inputs.

And there is the flaw in your logic. While it is true that searching all possibilities isn't feasible in practice due to upper time bounds of O(x^n), the space complexity for a depth first search is still O(n).

That joke puts it quite well. When in doubt, go for an recursive definition of the problem where you hold the additional dimensions as a list, and then go for a depth first solver. Time complexity becomes EXP, but you remain in PSPACE. Only way of getting out of PSAPCE would be to have a problem where even the number if dimensions varies at runtime, independent from the number of input or output dimensions.

I can only see 5 dims there: xyz,time and particle

It's actually 12 dimensions just for a SINGLE particle in a simple system with pure Newton mechanics. Coordinates, momentum, angular momentum, orientation, all of them having 3 components. That does not even include static attributes such as mass or a collision shape.

And then you need to multiply the number of particles with these dimensions to get the number of dimensions for expressing a single frame.

Dimensions is the number of distinct values you have which are required to express the system, in that case you would need to store a total of 24 dimensions, just for a system with 2 particles and extremely simplified physics. Add a basic set of other physical forces, and you get close to around 15-25 distinguishable and non-derivable properties or "dimensions" per particle involved.

Usually when you say "3D", it's just a shorthand for "I will consider at most 3 dimensions for each attribute type per object".
Even your most basic 3D graphics engine, which does not handle any physics or alike, expects about 6 or 9 dimensions (depending whether you want the ability to scale, or not) per object as an absolute minimum, and then add another bunch of immutable attributes such as vertex lists or textures.

I think the issue here is what you deem a dimension. For computational work it is a single attribute, however the broader definition would allow one dimension to include multiple attributes, meaning the example of the particle all exist within 3 dimensional space, potentially counting time as a 4th. neither view is wrong, its simply a case of using one term to describe different things, as always the key is context

@sorian I'm curious, any progress / new insights on the neural net overhaul? Or are you busy with fixing the load/save system as well?