Lifeboat Foundation: Safeguarding Humanity
Toggle light / dark theme

Blog

Category: computing – Page 184

Cerebras introduces gigaGPT: GPT-3 sized models in 565 lines of code.

GigaGPT is Cerebras’ implementation of Andrei Karpathy’s nanoGPT – the simplest and most compact code base to train and fine-tune GPT models. Whereas nanoGPT can train models in the 100M parameter range, gigaGPT trains models well over 100B parameters. We do this without introducing additional code or relying on third party frameworks – the entire repo is just 565 lines of code. Instead gigaGPT utilizes the large memory and compute capacity of Cerebras hardware to enable large scale training on vanilla torch.nn code. With no modifications, gigaGPT supports long context lengths and works with a variety of optimizers.

Why gigaGPT

While the transformer architecture is simple, training a large transformer on a large number of GPUs is hard. Beyond a few billion parameters, vanilla GPT models run out of memory on even the latest GPUs. Training larger models requires breaking up models into smaller pieces, distributing them to multiple GPUs, coordinating the workload among the workers, and assembling the results. This is typically done via LLM scaling frameworks such as Megatron, DeepSpeed, NeoX, Fairscale, and Mosaic Foundry. Though powerful, these frameworks introduce significant complexity.

Continue reading “Introducing gigaGPT: GPT-3 sized models in 565 lines of code” | >

Q-day (the day when quantum computers will successfully actually break the internet) may be some time away yet. However, that does not mean that companies — and states — shouldn’t hop on the qubit bandwagon now so as not to be left behind in the race for a technology that could potentially alter how we think about life, the Universe, and well… everything.

Spurred on by a discourse that more and more revolves around the concept of “digital sovereignty,” 11 EU member states this week signed the European Declaration on Quantum Technologies.

The signatories have agreed to align, coordinate, engage, support, monitor, and all those other international collaboration verbs, on various parts of the budding quantum technology ecosystem. They include France, Belgium, Croatia, Greece, Finland, Slovakia, Slovenia, Czech Republic, Malta, Estonia, and Spain. However, the coalition is still missing some quantum frontrunners, such as the Netherlands, Ireland, and Germany, who reportedly opted out due to the short time frame.

Read more | >

The team at EPFL’s Photonic Systems Laboratory (PHOSL) has developed a chip-scale laser source that enhances the performance of semiconductor lasers while enabling the generation of shorter wavelengths.

This pioneering work, led by Professor Camille Brès and postdoctoral researcher Marco Clementi from EPFL’s School of Engineering represents a significant advance in the field of photonics, with implications for telecommunications, metrology, and other high-precision applications.

The study, published in the journal Light: Science & Applications, reveals how the PHOSL researchers, in collaboration with the Laboratory of Photonics and Quantum Measurements, have successfully integrated semiconductor lasers with silicon nitride photonic circuits containing microresonators. This integration results in a hybrid device that emits highly uniform and precise light in both near-infrared and visible ranges, filling a technological gap that has long challenged the industry.

Read more | >

Physicists at RIKEN have developed an electronic device that hosts unusual states of matter, which could one day be useful for quantum computation.

When a material exists as an ultrathin layer—a mere one or a few atoms thick—it has totally different properties from thicker samples of the same material. That’s because confining electrons to a 2D plane gives rise to exotic states. Because of their flat dimensions and their broad compatibility with existing semiconductor technologies, such 2D materials are promising for harnessing new phenomenon in .

These states include quantum spin Hall insulators, which conduct electricity along their edges but are electrically insulating in their interiors. Such systems when coupled with superconductivity have been proposed as a route toward engineering topological superconducting states that have potential application in future topological quantum computers.

Read more | >

The negative side of not having it on a disk or flash drive.

Sony announced on Monday that it would remove all Discovery content, including shows like “MythBusters” and “Deadliest Catch,” from user libraries, even if they had been purchased on the PlayStation Store.

The company, which owns and operates PlayStation game consoles, said in a brief statement that the Discovery shows would be deleted on Dec. 31, attributing the decision to “our content licensing arrangements with content providers.”

The move came as Warner Bros. Discovery, which owns Discovery content, is seeking to add more subscribers to its Max and Discovery+ streaming services, in competition with others like Netflix and Disney+.

Continue reading “PlayStation Will Delete Purchased Discovery Shows” | >

Two-dimensional materials, which are only a few atoms thick, can exhibit some incredible properties, such as the ability to carry electric charge extremely efficiently, which could boost the performance of next-generation electronic devices.

However, integrating 2D materials into devices and systems like computer chips is notoriously difficult. These ultrathin structures can be damaged by conventional fabrication techniques, which often rely on the use of chemicals, high temperatures, or destructive processes like etching.

To overcome this challenge, researchers from MIT and elsewhere have developed a new technique to integrate 2D materials into devices in a single step while keeping the surfaces of the materials and the resulting interfaces pristine and free from defects.

Read more | >

IN THIS EPISODE OF THE HUMAN UPGRADE™…you’ll learn how much control you can actually have over your brain. There’s a new way of accessing your meat operating system that adds an important element that’s been missing from the brain training conversation. Moha Bensofia joins the show to explain a new piece of neurotech called Mendi. He shares the benefits of getting more blood to the front of your brain. Mendi uses neurofeedback to provide visual feedback based on your brain activity.

A headset measures the activity while you play a fun brain-training mobile game. The system uses fNIRS technology to measure blood flow and oxygenation in the pre-frontal cortex of the brain. The brain training game requires focus and calmness, which increases activity and control of your brain’s pathways.

Read more | >