Motivated by advances in the field of active matter where non-equilibrium forcing has been shown to activate new assembly pathways, here we study how non-equilibrium driving in prototypical memory formation models can affect their information processing capabilities. Our results reveal that activity can provide a new and surprisingly general way to dramatically improve the memory and information processing performance of the above described systems without the need for additional interactions or changes in connectivity. Non-equilibrium dynamics can allow these systems to have memory capacity, assembly or pattern recognition properties, and learning ability, in excess of their corresponding equilibrium counterparts. Counter-intuitively, in some cases, dynamics with non-equilibrium noise-sources can even have a higher memory capacity than zero temperature equilibrium systems that are not subject to any noise. Our results demonstrate the generality of the enhancement of memory capacity arising from non-equilibrium, active dynamics. These results are of significance to a variety of processes that take place under nonequilibrium dynamics, and involve information storage and retrieval, as well as in silico learning and memory forming systems for which nonequilibrium dynamics may provide an approach for modulating memory formation.