experimental navigation

Design Trend: Experimental Navigation Patterns

Long gone is the idea that navigation menus must be fixed at the top of a website design. While many designers opt for the safe, consistency of all caps navigation across the top of the screen with sans serif typography, more designs are breaking out of this pattern.

Experimental navigation patterns can be fun and interesting if they are intuitive enough for users to understand reasonably quickly. Different navigation styles can add interest to websites that are small, don’t have a lot of content or want users to move around in a specific way.

While experimental navigation isn’t for every design, it can be a fun alternative for the right project.

The Ultimate Designer Toolkit: 2 Million+ Assets

Envato Elements gives you unlimited access to 2 million+ pro design resources, themes, templates, photos, graphics and more. Everything you'll ever need in your design resource toolkit.

Presentation Templates

Powerpoint & keynote.

Icons, Vectors & More

Web Templates

Landing pages & email, side navigation.

There are plenty of ways to do side navigation. It can be static or dynamic and can be almost any width.

While it doesn’t seem like moving the nav from the top to the side is all that experimental, it can throw quite the wrench into your design because it changes the aspect ratio of the canvas.

Then you have to consider how this often-skinny navigation bar will look on other screen sizes. And what do you do if the navigation menu contains long words (you don’t want a nav bar filled with hyphens)?

There’s a lot to consider.

The best side navigation is simple, with short words and a fixed space. Too much scrolling in nav is jarring and difficult to understand. The example from Sanctum, above, is simple and clean. The nav stays in one place as the user scrolls, with the text adjusting only from light to dark as the background changes.

The nice thing about this example is that the vertical navigation pattern encourages users to first look at the name and logo and then move in s straight line down the screen to see what navigation options are available. It’s well-designed and functional.

Hidden and Pop-Out

One of the big things that has emerged from the prolific use of hamburger and other hidden icon-style navigation is pop-out menus.

Click or tap the button and the navigation swings out covering part or all of the screen (often depending on screen size).

In itself, this isn’t truly experimental. But the fact that so many designers are doing it so many different ways is. While users are arguably getting used to hamburger icons, these patterns are still somewhat unfamiliar. And with designers using varying icon types, there’s a little bit of a challenge there as well.

Nevertheless, the pop-out style from Caava Design, above, is interesting. Whereas most designers are going with flat, simple pop-out variations, this one has more depth to it. The design helps users find the most important parts of the navigation to lead them through the design.

Horizontal Scroll

The first few times you run across a horizontal scrolling site can be a little odd. It takes a specific design for this flow to feel right because of the strange difference in physical and visual movement.

To make this most of horizontal scrolling navigation, designers should use visual cues to help make the idea more comfortable for users. Arrows or other directional tools can be helpful.

Norgram, above, also uses a partial image as a visual cue that there is more content on the side of the screen with a fixed top to bottom look. The content is structured in such a way that the horizontal movement seems much more natural because of the visual cues provided.

No Navigation

Some websites are eliminating navigation altogether and opting for an everything on the screen style. It can be a tricky pattern for sure. Will users know what to click and what actions to take?

The “no navigation” navigation pattern works best for super small sites that are directing users to do one thing. It can work for a coming soon page or a thank you/recap style website such as the year in review page above. With just a handful of clickable items and a short scroll, it’s easy to figure out.

The simple animation in the design also helps. (You could arguably call it navigation because it encourages user movement). This can be a tricky pattern to say the least.

Single Page with Markers

Many of the experimental navigation patterns in use are being deployed on single-page websites. And for good reason: It is a lot harder for users to get lost in the one-page format.

To provide direction and help users feel like they are making progress in the design, many of these one-page navigation patterns rely on markers. Just like the traditional slider format with a dot or bar to note progress, this navigation style uses that same concept.

Socius, above, does a very nice job of this with markers on the right side of the page that include hover text that tells users what each dot represents. (This is a feature that’s often lacking in with this style of navigation.) Users can use the hover effect and dots to jump to specific information or scroll through the seven “screens.”

The trick to this style of design is to make everything feel quick. Slick scrolling effects and a digestible design, as shown in this example, help guide users through the content.

Subtle Edge Nav

Some designers are turning navigation 90 degrees and anchoring it to the right edge of the page. It’s a subtle trick that is primarily for small or portfolio-style websites that is popping up more commonly.

Navigation elements in this style tend to be text only, include just a handful of items and are generally small. The rotated navigational text can point into or out of the design, based on other elements on the screen.

Just like with vertical navigation, this idea can change the overall aspect ratio of the design because a sliver is cut from the side for navigation. The worry about this style is that navigational elements are subtle and small, making them easy to miss.

Are you more of a traditionalist when it comes to navigation, or are you willing to try something a little different? Experimental navigation patterns are one of those trends that seem to be gaining traction.

As more designers try these kinds of techniques, users get accustomed to the change and adapt. But there’s always that worry about users that “don’t get it.” I’d love to know what you think about different navigation styles. Let me know on Twitter and tag @designshack.

30+ Examples of Innovative and Experimental Navigation Experiences

Awwwards Magazine

In this article we want to go beyond the common and safe navigation patterns that provide an easy and natural way to access content. Everybody knows that the easiest way to navigate, in terms of usability and accessibility, is to use standard patterns and UI controls like tabs, off-canvas menu, navigation drawers, etc.

But this is Awwwards and we have a strong vocation of inspiration and experimentation, so we want to look further than the well known common solutions. The past few years had been dominated by the omnipresent parallax navigation effect , long scrolling websites and the scroll hijacking usability nightmare. Storytelling navigation has become very common too, and WebGL projects have come up with a lot of creative solutions to navigate content.

We are still waiting for more examples of gesture-driven interfaces , hopefully, the standarization of APIs accessing device hardware is bringing us all those longed for interaction experiences.

One step further in navigation is voice controlled interfaces , where a combination of speech recognition, natural language APIs and machine learning will show you great new ways to access content and delightful user experiences.

Finally we would like to mention the new “black” in user experience design topics: VR and AR. Navigation is one of the most difficult tasks to accomplish in virtual reality scenarios. We need to go beyond Gaze and teleport navigation. Fortunately this is a topic we have to deal with now because VR and AR are no longer a future dream.

Here we look at a selection of sites which implement different navigation patterns and techniques like timeline or scroll triggered navigation, infinite canvas, parallax and storytelling navigation. You can find more in our collection The Best of Navigation . We will keep adding to it in order to provide you with an up-to-date reference.

Creative Navigation Experiences

Head to our blog article to see these examples and many more in motion.

If you have liked our selection of Navigation elements and experiences, why not add it to your favorites? Did you know you can follow your favorite collections (which are continually updated to provide you with the most up-to- date inspiration) or you can even create your own. To see how, follow this simple YouTube tutorial and keep your eye out for the new Chrome extension that will be available in the next few weeks.

Originally published at www.awwwards.com .

Written by Awwwards

The awards for design, creativity and innovation on the Internet, which recognize and promote the best web designers in the world

Text to speech

How experimental web design can take work in new directions

Learn the fundamentals and see examples of great experimental web designs.

Build completely custom, production-ready websites — or ultra-high-fidelity prototypes — without writing a line of code. Only with Webflow.

Experimental web design lets us break free from constraints.

As designers, we often play it safe. We drag in a few layout elements and set up navigational options the same way we’ve done countless times before. And there’s nothing wrong with sticking to popular web page layouts — after all, they’ve been proven to work. 

But design can be so much more. Let’s take a look at experimental web design and explore some examples that might inspire you to step out of your own comfort zone.

Experimentalism defies traditional layouts

Single column, z-pattern, split-screen, and other arrangements are plentiful because they’ve proven their functionality. We don’t have to think about how to navigate these layouts because we’re already accustomed to them.

Experimental web design takes away from the familiar. Distinct lines disappear, creating a somewhat jarring first impression. The perfection of grids is eschewed for a looser, more imaginative approach that makes for a very different user experience than what we’re used to. 

Experimental UI design

Traditional UI design centers around familiar, intuitive navigation. Experimental web design intentionally disrupts this routine flow. At first, site visitors might be a little unsure of what to do, like when starting a puzzle. But once visitors get used to the logic behind an experimental user interface, they can explore all that the website has to offer. 

Every day, we perform actions that are so routine that we barely register that we’re doing them. We don’t give any more thought to scrolling on a website than we do to pushing a button on an elevator. We go through the movements automatically. Experimental web design challenges us to work outside of those instincts. 

A site with experimental design may ask us to scroll through its content horizontally. It may direct us to do something like hold down the space bar to explore the content or provide other unorthodox UI guidance. While these tactics can be employed playfully, it’s important to recognize they’ll make a website significantly less accessible. Building a website with accessibility and experimentation in mind can be a challenge, but it is possible. If you do upend traditional UI choices, make sure you provide direction on how to use or circumvent them, and that those directions are accessible to all of your site visitors.

This space-themed website for Stykovka completely disregards navigational norms. Instead, it includes clear instructions for how to interact with the site. Experimentalism often works outside of the standards we’re used to, so the guidance from Stykovka is helpful and necessary. 

Experimental design explores different ways to work with visuals

Traditional websites typically don’t shake things up in terms of their visual experiences. Experimental design is full of surprises. If web designs were sculptures, experimentalism is like the chisel that carves out spaces and forms that defy conventionality. 

While experimentalism means user experiences and how ideas are communicated may take a radical departure from tradition, it’s important to remember to use it only when it’s additive to the message or form. You wouldn’t make a bank homepage experimental, but you might do so for an interactive landing page that’s designed to pull a visitor in.

Here are some implementations of effective experimental visual design.

Virtual reality offers surreal three-dimensional spaces

Zendesk’s Museum of Annoying Experiences offers a lighthearted virtual reality journey into the minor frustrations of modern life.

Web designers are breaking free from the conventions of grids, text, and static graphics by  creating three-dimensional worlds begging to be explored. Virtual reality websites offer a sense of immediacy and interactivity not possible through traditional web design. 

Data visualizations display data in creative ways

The web had opened up new realms in representing data. Data visualizations transform numbers and analytics by presenting them in ways that are more interesting and engaging. 

At digital publication The Pudding data visualization is the medium. They use engaging experimental visualization to cover topics that may span long periods of time, or large data sets.

Experimental web design often takes great liberties in its visualizations and shares data in new and imaginative ways. 

Integrating new advances in tech in web design

New technologies have a long history of shaping design. Printing presses changed how books were made. Computers gave graphic designers more freedom to explore their craft. Advances and experimentation in technology are often linked to evolution in art and design. 

Now that machine learning and artificial intelligence (AI) are becoming more accessible,  designers are finding ways to apply these technologies to web development. Things like predictive UX can provide a more efficient and personalized experience. Machine learning can power algorithms for product recommendations. 

AI and machine learning can also be harnessed to create a richer visual experience. Visual gesture recognition systems that track a visitor’s movements provide a novel way of interacting with a website, like this site that teaches ASL Fingerspelling . 

Bitmoji-styled graphics give visitors a fun and personalized representation of themselves, generated in part by AI. We’re only at the beginning of seeing how artificial intelligence and machine learning can be used to add to the visual nature of web design.

Discover the processes and tools behind high-performing websites in this free ebook.

Experimental web design opens up new possibilities

Now that we’ve discussed some of the principles of experimental web design, let’s take a more in-depth look at a few website designs that are anything but traditional. You’ll see that there’s no single flavor of experimentalism, with each of these websites taking vastly different approaches.

Project Turntable

We’ve talked about the agency ThreeSixtyEight before when we shared how they transitioned from WordPress to Webflow . They’re a top-notch outfit full of imaginative and innovative design ideas.

This landing page , which exists in addition to their main website, offers a hi-fi inspired experience. At the top of the screen, an arrow directs visitors to the play/standby button. When the switch is flipped, the record starts spinning a 60s-inspired groove. 

The interactive analog buttons and switches set this design apart. ThreeSixtyEight uses experimental web design to celebrate their creative sensibilities while giving their visitors something different from what other agencies are doing.

Reminiscence Movie

This website is for the fictional agency Bannister and Associates, which is part of the universe created in the sci-fi noir film Reminiscence. Even if you haven’t watched the film, this website is worth checking out.

The site uses AI to create “memories” for its visitors. Users are prompted to upload an image of themselves or of someone significant in their lives. Each step then takes them through the process of generating a memory. 

At the final screen you see a floating formation of orbs and you’re prompted to connect the synapses. The image that you’ve uploaded is then shown to you, with the face brought to life with motion. It’s strange and unsettling, but a lot of fun.

This is a great example of how AI can be integrated into a website, giving visitors a unique and engaging visual experience not possible in more conventional designs.

The Pudding

We appreciate The Pudding (yes, we’ve mentioned them twice in this article, but we’re just SUCH big fans) for their flair for telling interesting and compelling stories through visuals. Whether they’re analyzing the comedy of Ali Wong, documenting wrestling masks, or putting together the definitive internet boy band database — they’re always finding novel ways to present information.

This chart shows musical artists who’ve gone solo, documenting their followers on Spotify compared to the bands they were formerly members of. This interactive chart is more appealing than a regular graph and makes the information immediately understandable.

Experimental web design offers new ways of presenting data. The Pudding excels at putting together data visualizations that not only communicate greater ideas but also do so in a visually engaging way.

Music Theater Showcase

Built with Webflow , this website for Baldwin Wallace University’s theater department steps out of the confines of stuffy academic design to create a website that captures the essence of acting.

Navigation happens through a spotlight that guides us through the space, lighting up what’s ahead. We’re brought to the center stage, where we get to find out more about the students who have graduated from the program. 

Instead of the same grids and columns we see again and again, this site takes us into the world of theater. 

When we scroll through Krivitzy , a Russian law firm’s website, we’re treated to graphics and animations that look like they’re straight out of an edgy comic book. 

For the section on bankruptcy, we see a man tied to a diamond, who then rockets away via a jetpack. A demon appears on the page for tax disputes. A fiery dragon is transformed into a docile rabbit under the wand that represents the law firm. Many other odd visuals pop up as we navigate the site. 

We don’t often associate the legal field with pushing creative boundaries. But experimental web design shows that whatever the field, there are ways to tell a brand’s story in unconventional ways.

Stonewall Forever

The Stonewall Riot is considered the flashpoint of the LGBTQ+ movement. This virtual reality journey tells its story through dazzling visuals.

Web designers once had a limited palette. Two-dimensional images, columns, rows, and text were all they had to communicate ideas or to tell a story. Virtual reality has opened up wide creative expanses, giving designers the power to communicate in ways that are more immediate and vibrant.

Black dog story

We often get caught up with the more commercial aspects of web design. But there’s much that can be done in this visual medium that doesn’t have to serve in marketing products or companies. Black Dog Story embraces all of the possibilities of web design, using interactions, animations, and other visuals to tell a story in a way that feels like fine art.

A pulsating ellipse beckons one to click through each frame of this story, rendered in beautiful blacks and greys. Shapes swirl and morph, and we’re taken along what feels like a beautiful dream.

Where most architectural firms display a collection of projects they’ve worked on in a grid. Bivak takes an interesting side-scrolling approach. Instead of a classic carousel image slider, Bivak uses horizontal scrolling to take us to each project page. 

Bivak could have stayed safe with standard top or navigational options. Instead, they’ve taken an experimental approach that relates to who they are as a modern architectural firm.

Webflow can set you free

We love seeing designers pushing their work in fresh and imaginative directions. 

Often the barrier to creating something unique lies with the tools you have to work with. We’re happy to offer an intuitive visual design platform that can help you get your idea out there no matter how wild or experimental. 

If you need any more inspiration, check out Made in Webflow , where designers are continually posting their latest projects. Or sign up for Webflow Inspo , our newsletter where we share the most incredible Webflow projects each week.

Subscribe to Webflow Inspo

Get the best, coolest, and latest in design and no-code delivered to your inbox each week.

Related articles

18 web design trends for 2017

Discover the web design trends that will define website and digital product design in 2017 — and beyond.

17 web design trends for 2016

Discover what's shaping the world of web design today — and tomorrow.

Website personalization: What it is & how to get started

Learn how adding website personalization to your marketing strategy results in more conversions and a better user experience for your website visitors.

11 font generator tools

Check out these 11 typography tools to help you with your web designs.

How fintech marketing teams can make web design work for them

Hoping to win over fintech customers? Learn how to leverage web design to create seamless experiences and streamline your marketing efforts.

A deep dive into brutalism

An overview of brutalism including its origins, key features, and examples of brutalist web design.

Get started for free

Try Webflow for as long as you like with our free Starter plan. Purchase a paid Site plan to publish, host, and unlock additional features.

Transforming the design process at

• Interactions
• Localization
• DevLink Labs
• Figma to Webflow Labs
• Accessibility
• Global alliances
• Freelancers
• Customer stories
• Webinars and ebooks
• Libraries Beta
• Made in Webflow
• Livestreams
• Careers We're Hiring
• Webflow Shop
• Terms of Service
• Privacy policy
• Cookie policy
• Cookie preferences
• Accessibility statement
• Discover the community
• Partner with Webflow
• Certified partners
• Become a Template Designer
• Become an Affiliate
• Become a global leader
• Find a meetup near you

Experimental Navigation: Should You Do It?

Traditional navigation menus have served us well, but that hasn’t stopped designers from pushing boundaries to deliver innovative and engaging user journeys. Let’s dive into the realm of experimental navigation and explore whether or not you should incorporate it into your website.

The Purpose of Navigation

Navigation is a fundamental component of any website, serving as a roadmap for users to explore its content and functionality. Traditional navigation menus, often located at the top or side of a webpage, present a hierarchical structure with clearly labeled categories and subcategories. While this approach is widely accepted and understood, experimental navigation aims to break free from convention to create a more memorable and interactive user experience .  

Check out our work with Movement Strategy.

Experimental site navigation can be a great way to break the mold and create a unique user experience. However, it’s important to thoroughly test designs before implementing them. If you’re not careful, experimental navigation can actually alienate users and make it harder for them to find what they’re looking for. The key to successful non-traditional navigation is to make sure that it’s still intuitive and easy to use. If users have to spend too much time figuring out how to navigate your site, they’re going to get frustrated and leave.”

                  – Kirill Davis, Big Drop’s UX/UI Lead 

The Pros of Experimental Navigation

Differentiation and Brand Identity: Experimental site navigation allows your website to stand out from the crowd. By challenging the norm, you have the opportunity to create a unique visual identity that aligns with your brand and captures users’ attention.

Check out our work with Treehouse .

Enhanced User Engagement: Experimental navigation techniques, such as hidden menus, animated elements, or unconventional placement, can spark curiosity and encourage users to explore further. It can make the navigation process feel more like an adventure, fostering a sense of excitement and engagement.

Memorable User Experience: Websites that implement experimental navigation often leave a lasting impression on visitors. By offering a novel and unexpected interaction, users are more likely to remember your website and return for future visits.

Check out our work with Level One Fund .

Showcase Innovation: If your website represents a cutting-edge product or service, experimental navigation can be an effective way to convey innovation. It aligns your design with your brand’s forward-thinking mindset and demonstrates your willingness to push boundaries.

Usability and Accessibility: While experimental navigation can be visually appealing, it’s crucial to maintain a balance between creativity and usability. Consider the needs of your target audience and ensure that your navigation remains intuitive and accessible to all users. Conduct thorough user testing to gather feedback and make improvements based on real-world usage.

Consistency: Experimental site navigation should not sacrifice consistency across your website. Maintain a coherent user experience by ensuring that users can still find their way around your website easily, regardless of the innovative elements you introduce.

Performance Optimization: Some navigation techniques, particularly those involving complex animations or interactions, can negatively impact website performance. Take care to optimize your website’s speed and performance, as slow-loading pages can frustrate users and lead to higher bounce rates.

Mobile Responsiveness: As mobile browsing continues to dominate, it’s essential to ensure that your experimental navigation translates well across different devices and screen sizes. Test your design thoroughly on mobile devices to ensure a seamless experience.

In Conclusion

Experimental navigation offers an exciting opportunity to break away from conventional design patterns and create a truly unique user experience on your website. By embracing innovation, you can differentiate your brand, increase user engagement, and leave a lasting impression. However, it’s important to balance creativity with usability and maintain consistency while optimizing performance and prioritizing mobile responsiveness. Ultimately, the decision to incorporate experimental site navigation should be based on your brand identity, target audience, and the goals you aim to achieve with your website. Remember to test and iterate based on user feedback to ensure a positive experience for all visitors. And if you need any assistance, you can always reach out to us here at Big Drop .

Website Design for Healthcare

All about qa, asset selection: how photography style can make or break your website, want to collaborate on a future-forward project.

We collaborate with innovative brands to create cutting-edge digital experiences. Let's connect.

Pentagon gearing up to launch 1st experimental navigation satellite in 50 years (video)

'It's probably going to affect so many lives.'

The Pentagon is preparing to launch a test craft that could improve the nation's satellite-navigation capabilities.  

The spacecraft, called Navigation Technology Satellite-3 (NTS-3), will be the first U.S. experimental navigation satellite to take flight since NTS-1 and NTS-2 launched in the 1970s.

NTS-3, which was developed by L3Harris Technologies, recently completed testing at the Benefield Anechoic Facility (BAF) at Edwards Air Force Base in California. The specialized facility enabled technicians to test the satellite's systems without interfering with or inadvertently jamming nearby air, auto and other GPS (global positioning system) signals. 

Related: How GPS systems help people navigate

The novel technology onboard NTS-3 is designed as a response to the increasing potential for threats to existing satellite-navigation constellations . According to a press release from the U.S. Space Force , "resilient approaches to augment the GPS system are needed to maintain users' access." The new satellite will also test technologies that allow GPS connections to be maintained in military conflict zones. 

NTS-3 is going to "help with all of our GPS systems," Amarachi Egbuziem-Ciolkosz, an engineer with the 772nd Test Squadron, said in the press release . "It's probably going to affect so many lives, not just military but commercial alike." 

The BAF is the largest anechoic testing site in the world, but this was the facility's first test of a satellite in decades. 

Get the Space.com Newsletter

Breaking space news, the latest updates on rocket launches, skywatching events and more!

"It is a quiet enough chamber that protects aircraft and other GPS users outside of the facility," NTS-3 program manager Ariel Biersgreen said in the same statement. "We needed to have a shielded area large enough to keep the energy of the testing inside the facility. Across the board, the BAF fit the bill in a way no other facility in the United States really could." 

"Whether it be airline schedules, takeoffs and landings at airports or military operations, NTS-3 is taking this a step further, because we are doing a demonstration of advanced signals and signal flexibility," added NTS-3 chief engineer Thomas Roberts. "Our ability to get that job done is dependent on the success of this testing facility."

— This device will make GPS work on the moon

— How Russia's GPS satellite signal jamming works, and what we can do about it

— China launches BeiDou navigation satellite to orbit (video)

NTS-3 is on track to launch near the end of this year. Before that happens, however, it must undergo thermal vacuum tests at Kirtland Air Force Base in New Mexico. There, the satellite will be exposed to a simulated space environment to ensure its ability to function as expected once in orbit. NTS-3 will operate in a near-geosynchronous orbit for approximately one year, if all goes to plan. 

According to the press release, the recent successful testing at BAF was a collaborative effort of the U.S. Air Force and Air Force Research Lab, the Space Force and NASA's Jet Propulsion Laboratory , all of whom "had critical roles in this historic testing of the NTS-3 satellite."

Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: [email protected].

Josh Dinner is Space.com's Content Manager. He is a writer and photographer with a passion for science and space exploration, and has been working the space beat since 2016. Josh has covered the evolution of NASA's commercial spaceflight partnerships, from early Dragon and Cygnus cargo missions to the ongoing development and launches of crewed missions from the Space Coast, as well as NASA science missions and more. He also enjoys building 1:144 scale models of rockets and human-flown spacecraft. Find some of Josh's launch photography on Instagram and his website , and follow him on Twitter , where he mostly posts in haiku.

International Space Station: Live updates

Collins backs out of ISS spacesuit contract with NASA

Japan launches advanced Earth-observing satellite on 3rd flight of H3 rocket

Most Popular

• 2 Mars Odyssey celebrates 100,000 orbits, captures epic view of solar system's largest volcano
• 3 It's International Asteroid Day, and astronomers have much to celebrate
• 4 The new space race: International partnerships (op-ed)
• 5 Ugreen PowerRoam 2200 power station review

Experimental Navigation

Inspirational designs, illustrations, and graphic elements from the world’s best designers. want more inspiration browse our search results ....

• For designers
• Hire talent
• Inspiration
• Advertising
• © 2024 Dribbble
• Freelancers

• navigation_experimental

ROS 2 Documentation

The ROS Wiki is for ROS 1. Are you using ROS 2 ( Humble , Iron , or Rolling )? Check out the ROS 2 Project Documentation Package specific documentation can be found on index.ros.org

• Distributions
• ROS/Installation
• ROS/Tutorials
• RecentChanges
• Immutable Page
• Attachments

Documentation

Report a bug.

Wiki: navigation_experimental (last edited 2018-09-03 09:27:56 by MartinGuenther )

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

ROS Navigation stack. Code for finding where the robot is and how it can get somewhere else.

ros-planning/navigation

Folders and files.

Repository files navigation

Ros navigation stack.

A 2D navigation stack that takes in information from odometry, sensor streams, and a goal pose and outputs safe velocity commands that are sent to a mobile base.

Related stacks:

• http://github.com/ros-planning/navigation_msgs (new in Jade+)
• http://github.com/ros-planning/navigation_tutorials
• http://github.com/ros-planning/navigation_experimental

For discussion, please check out the https://groups.google.com/group/ros-sig-navigation mailing list.

Contributors 125

• Python 2.8%

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 14 November 2023

Multisensory input modulates memory-guided spatial navigation in humans

• Deetje Iggena   ORCID: orcid.org/0000-0001-8778-5127 1 , 2   na1 ,
• Sein Jeung 3 , 4 , 5   na1 ,
• Patrizia M. Maier 1 , 2 ,
• Christoph J. Ploner 1 ,
• Klaus Gramann   ORCID: orcid.org/0000-0003-2673-1832 3 , 6 &
• Carsten Finke   ORCID: orcid.org/0000-0002-7665-1171 1 , 2  

Communications Biology volume  6 , Article number:  1167 ( 2023 ) Cite this article

2328 Accesses

3 Citations

5 Altmetric

Metrics details

• Hippocampus
• Human behaviour
• Spatial memory

Efficient navigation is supported by a cognitive map of space. The hippocampus plays a key role for this map by linking multimodal sensory information with spatial memory representations. However, in human navigation studies, the full range of sensory information is often unavailable due to the stationarity of experimental setups. We investigated the contribution of multisensory information to memory-guided spatial navigation by presenting a virtual version of the Morris water maze on a screen and in an immersive mobile virtual reality setup. Patients with hippocampal lesions and matched controls navigated to memorized object locations in relation to surrounding landmarks. Our results show that availability of multisensory input improves memory-guided spatial navigation in both groups. It has distinct effects on navigational behaviour, with greater improvement in spatial memory performance in patients. We conclude that congruent multisensory information shifts computations to extrahippocampal areas that support spatial navigation and compensates for spatial navigation deficits.

Similar content being viewed by others

Behavior-dependent directional tuning in the human visual-navigation network

Remapping and realignment in the human hippocampal formation predict context-dependent spatial behavior

Linking hippocampal multiplexed tuning, Hebbian plasticity and navigation

Introduction.

The ability to navigate distinct environments and locations from memory is a prerequisite for autonomy and survival. For effective navigation, we continuously update our position and orientation by integrating multisensory information with memory representations of the environment 1 , 2 , 3 . Relevant sensory information includes visual, vestibular, and proprioceptive input 4 , 5 , 6 , 7 , 8 . These inputs are integrated and transformed into spatial representations, which are encoded, consolidated, and eventually recalled, in formats that depend on actual behavioral demands 9 , 10 .

One of the core regions in brain networks for spatial navigation is the hippocampus. The hippocampus binds and integrates location-specific information from multiple sensory modalities and uses it to transform spatial relationships into a global cognitive map 11 , 12 . During the formation process of this map, the hippocampus is in constant exchange with other brain regions and shares computations with the parahippocampal, entorhinal, and the retrosplenial cortex, among others 1 , 13 . Accordingly, behavioral assessment of spatial navigation has become an important tool to test hippocampal function across species. In animal models, particularly in rodents, navigation in environments such as the Morris Water Maze (MWM) is a widely used standard in spatial memory research 14 , 15 . However, in many human navigation experiments, memory-guided spatial navigation is mainly investigated with moving visual stimuli on a stationary screen, without the vestibular, and proprioceptive inputs that contribute to behavior in animal experiments. The absence of these body-based sensory inputs creates an artificial situation that limits spatial information for navigation and may promote behaviors that do not necessarily reflect everyday demands in human participants 16 , 17 . In particular, for humans with structural or functional lesions in brain regions critical for spatial navigation, alternative behaviors may be triggered depending on the availability of body-based sensory cues 18 , 19 , 20 , 21 . The ecological validity of stationary navigation paradigms has therefore been repeatedly questioned 22 , 23 , 24 .

Advances in mobile immersive virtual reality (VR) technologies provide an opportunity to overcome these limitations and to study human spatial navigation with true multisensory input 25 , 26 , 27 . Head-mounted displays (HMDs) immerse participants in a highly realistic yet controlled virtual environment, in which they can move freely, generating and receiving ample body-based sensory input. Mobile VR systems, therefore enable the test conditions for humans that are largely analogous to animal experiments. This allows a more direct comparison of data from humans with data from freely moving rodents obtained in navigation tasks such as the MWM.

The present study aimed to systematically investigate the contribution of multisensory information to human spatial navigation. We asked whether and how memory-guided spatial navigation benefits from multisensory input in humans with and without hippocampal dysfunction. Particularly, we were interested in how hippocampal lesions alter the use of multisensory information and whether this is reflected in altered navigational behavior. To this end, we implemented a virtual MWM task and compared the navigation patterns of patients with hippocampal lesions to those of healthy controls in both a stationary desktop and in a room-scale mobile VR environment.

We found that the availability of multisensory information improved memory-guided spatial navigation in both patients and healthy controls, with greater improvement in spatial memory performance in patients. This improved memory performance was accompanied by different navigation behavior in patients and healthy controls. This suggests that multisensory information may shift computations to extrahippocampal areas that support spatial navigation and compensate for hippocampus-related deficits in spatial navigation.

To evaluate the effects of multisensory input on memory-guided spatial navigation, we examined spatial memory performance and navigation behavior in a virtual version of the MWM. While a water maze, is typically implemented in rodent studies in a pool filled with water, we built a dry version of human scale water maze, using a virtual circular enclosure filled with virtual ground fog. It was presented either on a screen where participants navigated the environment with a joystick (stationary), or in a room-scale immersive VR setup where participants could walk freely during the task (mobile), (Fig.  1a, b , see methods). The session order of experimental setups was counterbalanced to account for potential learning effects from the first experimental setup that could influence navigation behavior in the second experimental setup. The circular arena of the water maze was surrounded by landmarks embedded in a natural-looking hilly landscape. We counterbalanced the design of the environment between the two experimental setups stationary and mobile (Fig.  1c ).

a Exemplary images of stationary setup where the participant stands in front of a screen. b Exemplary images of mobile setup where the participant wears head-mounted display glasses to experience VR. c Exemplary participant view of the virtual environments. Two different scenes were created for counterbalanced design across the two experimental setups stationary and mobile . d Experimental block design. Each block started with three learning trials in which the starting location remained constant, and the object appeared as feedback for participants. Learning trials were followed by four probe trials, in which the starting locations varied with rotations around the center by 0, 90, 180, or 270°, and participants had to indicate the remembered object location. A total of six object locations were learned in each setup. All consecutive trial pairs were separated by a spatial disorientation task.

Patients with hippocampal lesions (MTLR, n  = 10) and their healthy controls (Control, n  = 20) learned the locations of objects by exploring the water maze, repeatedly starting from the same location. During three learning trials, the object appeared as soon as the target location was reached (Fig.  1d ). After the three learning trials, four probe trials followed from four different starting locations to encourage the use of spatial relations in the environment. In probe trials, the object remained hidden, and participants indicated where they remembered the hidden object by pressing a button. Per experimental setup, six target objects were placed at different distances to the boundary to discourage the use of a circling strategy around the arena and in a distinct angular relation to the landmarks, to promote triangulation between landmarks and the target locations (Supplementary Table  1 )

After each learning or probe trial, a disorientation task followed that forced self-localization at the onset of each trial in both stationary and mobile setups. Briefly, in this task, all spatial cues were blanked out. Participants first navigated to spheres that triggered a random sequence of three turns, then they were led by spheres to the starting point of the next trial and the spatial features of the virtual environment reappeared (see methods and Supplementary methods  1 ). By computing how close the final response location was to the target location, we assessed spatial memory performance, and by inferring behavioral patterns from the path traveled, we analyzed navigation efficiency and navigation strategies (see methods).

Multisensory input improves spatial memory performance in patients with hippocampal lesions

We investigated how accurately participants retrieved the learned object locations during probe trials. This aspect of spatial navigation depends on the integrity of spatial memory representations and the ability to determine one’s location relative to the environment.

We calculated the distance between the final location of the participants and the actual target location to derive a memory score, taking into account the geometry of the environmental boundary 28 , 29 , (Fig.  2a ). The memory score ranges from 0 to 100%, with values close to 100% suggesting that the participant could perfectly remember and locate the target location, 50% suggesting that the final location was chosen at chance level, and close to 0% that the participant’s final position was systematically biased in the opposite direction of the actual target location (see methods).

a Schematic representation of the calculation of the memory score. 1000 random locations with uniform spatial distribution were generated. The percentage of locations with smaller distance to the target location than the participant’s final chosen location was subtracted from 100%. 50% corresponds to random-level performance, a higher memory score indicates a bias towards the target location and a score below 50% a bias towards the opposite direction of the target location. b The memory score as a measure of spatial memory performance. Patients benefited more from access to multisensory information and showed a greater increase in memory performance than the control group (setup*group: F (1,27)  = 5.207, p  = 0.031, ω 2  = 0.13). c Schematic representation of the calculation of the scatter of the final locations. The distance between each final location pair was averaged across all distances per target location. A value closer to zero indicates less scatter of the final locations and thus higher spatial precision. d Scatter of final locations as a measure of spatial precision. Patients benefited more from access to multisensory information and showed a greater decrease in scatter of final locations than the control group (setup*group: F (1,52)  = 9.595, p  = 0.003, ω 2  = 0.14) e Final locations in the arena are shown as a percentage at each location. Yellow indicates that more than half of the responses occurred at that location; dark blue indicates that no responses occurred at that location. Target location is marked with a white circle. Metric data presented as boxplots with a center line as median, Tukey-style whiskers extend 1.5 times the interquartile range from the 25th and 75th percentiles. Dots present individual datapoints. Data was analyzed with a linear mixed model. Sample size, medial temporal lobe resection (MTLR) group: n  = 10, control: n  = 20; ∗  =  p  ≤ 0 .05; ∗ ∗  =  p  ≤ 0.01; ∗ ∗ ∗  =  p  ≤ 0.001.

Both patients with medial temporal lobe resections including the hippocampus and healthy controls, performed above chance level in both the stationary desktop and the mobile VR setup (Fig.  2b ). In the stationary setup, patients had a lower average memory score compared with their healthy controls (Mean ± SEM: 72.4 ± 5.4 vs. 87.6 ± 2.1; Supplementary Table  2 ). However, when participants had access to multisensory information in the mobile setup, the memory score increased by 23.6% in patients (Mean ± SEM: 72.4 ± 5.4 to 89.5 ± 1.8) and by 7.6% in controls (Mean ± SEM: 87.6 ± 2.1 vs. 94.6 ± 1.0). Although memory performance of both groups benefited from multisensory information, the change in memory score across setups was significantly more pronounced in patients compared to healthy controls (setup*group: F (1,27)  = 5.207, p  = 0.031, ω 2  = 0.13; stat-MTLR vs. mobile-MTLR, p  < 0.001; stat-control vs. mobile-control, p  = 0.031; stat-MTLR vs. stat-control, p  < 0.001; mobile-MTLR vs. mobile-control, p  = 0.251).

Multisensory input improves spatial precision in patients with hippocampal lesions

We then investigated the precision of the representations underlying memory-guided navigation, a property that also depends on hippocampal integrity 30 , 31 . Regardless of the distance to the target location, precision indicates how consistent responses were per target location. As a measure of precision, we computed the scatter of participants’ responses by calculating the relative distance of all six distances between the four final locations per target location in the probe trials. A smaller scatter meant higher spatial precision (Fig.  2c–e , see methods).

In the stationary setup, patients showed a higher average scatter in final locations compared with their healthy controls (Mean ± SEM: 0.23 ± 0.03 vs. 0.12 ± 0.01; Supplementary Table  2 ). When multisensory input was available in the mobile setup, the scatter decreased in both groups, by 59.9% in patients (Mean ± SEM: 0.23 ± 0.03 vs. 0.09 ± 0.01) and by 48.7% in controls (Mean ± SEM: 0.12 ± 0.01 vs. 0.06 ± 0.00). As with the memory score, the change in spatial precision was significantly more pronounced in patients compared to healthy controls (setup*group: F (1,52)  = 9.595, p  = 0.003, ω 2  = 0.14; stat-MTLR vs. mobile-MTLR, p  < 0.001; stat-control vs. mobile-control, p  = 0.002; stat-MTLR vs. stat-control, p  < 0.001; mobile-MTLR vs. mobile-control, p  = 0.194).

Multisensory input improves spatial navigation efficiency in patients with hippocampal lesions

Analysis of navigation path to a location reveals various behavioral properties, such as the temporal and spatial efficiency of navigation. The better one can locate themselves and the target location in relation to landmarks, the faster and more directly the targeted destination can be reached.

The temporal efficiency of navigation can be assessed by the latency to the final location (Fig.  3a, b , see methods). We found that temporal efficiency increased in both, patients and healthy controls when multisensory input was available in the mobile VR setup. In learning trials, the improved temporal efficiency in the mobile setup compared to the stationary setup was reflected in a reduction in average latency to final location by 48.8% in patients (Mean ± SEM: 46.7 ± 17.4 vs. 23.9 ± 4.9, Supplementary Table  3 ) and by 22.2% in controls (Mean ± SEM: 19.8 ± 2.3 vs. 15.4 ± 1.1; setup: F (1,27)  = 7.310, p  = 0.012, ω 2  = 0.18). Across experimental setups, patients required more time to reach the final location than the healthy controls in the learning trials (group: F (1,25)  = 6.457, p  = 0. 018, ω 2  = 0.17). In probe trials, the improved temporal efficiency in the mobile setup was reflected in a reduction in latency by 36.4% in patients (Mean ± SEM: 25.8 ± 2.7 vs. 16.4 ± 1.6) and by 34.5% in controls (Mean ± SEM: 27.5 ± 1.8 vs. 18.0 ± 1.4; setup: F (1,27)  = 52.153, p  < 0.001, ω 2  = 0.64). Across experimental setups, patients had a similar latency to reach the final locations as controls in probe trials (group: F (1,25)  = 0.664, p  = 0. 423, ω 2  = 0.0).

a Latency to final location as measure of temporal efficiency in learning trials. Multisensory input in the mobile setup increased temporal efficiency for both groups, as evidenced by reduced latency to final location (setup: F (1,27)  = 7.310, p  = 0.012, ω 2  = 0. 18). b Latency to final location as measure of temporal efficiency in probe trials. Multisensory input in the mobile setup increased temporal efficiency for both groups, as evidenced by reduced latency to final location (setup: F (1,27)  = 52.153, p  < 0.001, ω 2  = 0. 64). c Path error to final location as measure of spatial efficiency in learning trials. Multisensory input in the mobile setup increased spatial efficiency for both groups, as evidenced by reduced path error (setup: F (1,52)  = 7.897, p  = 0.007, ω 2  = 0.11). d Surface coverage as a measure of spatial efficiency in learning trials. Multisensory input in the mobile setup increased spatial efficiency for both groups, as evidenced by reduced path error (setup: F (1,27)  = 33.499, p  < 0.001, ω 2  = 0. 53). e Path error to final location as measure of spatial efficiency in probe trials. Multisensory input in the mobile setup increased spatial efficiency for both groups, as evidenced by reduced path error (setup: F (1,27)  = 48.153, p  < 0.001, ω 2  = 0. 62). f Surface coverage as measure of spatial efficiency in probe trials. Multisensory input in the mobile setup increased spatial efficiency for both groups, as evidenced by reduced path error (setup: F (1,27)  = 26.654, p  < 0.001, ω 2  = 0. 47). g , h Presence probability in learning trials and probe trials. Presence probability describes the probability of being in a cell of a 20*20 grid covering the arena surface. Yellow means that the paths passed through that location in more than half of the trials, dark blue means that the paths did not pass the underlying cell of the grid at all. Target location is marked with a white circle. Metric data presented as boxplots with a center line as median, Tukey-style whiskers extend 1.5 times the interquartile range from 25 th and 75 th percentiles. Dots present individual datapoints. Data was analyzed with a linear mixed model. Sample size, MTRL: n  = 10, control: n  = 20; ∗   =  p  ≤ 0 .05; ∗ ∗  =  p  ≤ 0.01; ∗ ∗ ∗  =  p  ≤ 0.001.

Spatial efficiency is reflected in the path error and surface coverage. The path error is calculated as the percentage of deviation of the actual path from an ideal path to the final location, and surface coverage is determined by the percentage of the arena area covered during navigation (Fig.  3c–f , see methods). As with temporal efficiency, we found that spatial efficiency increased in both, patients and healthy controls when multisensory input was available in the mobile VR setup (Fig.  3g, h ). In learning trials, the improved spatial efficiency in the mobile setup was reflected in a decrease in average path error by 69.7% in patients (Mean ± SEM: 745.5 ± 175.5 vs. 226.1 ± 65.1) and by 67.9% in controls (Mean ± SEM: 530.6 ± 207.3 vs. 170.5 ± 21.0; setup: F (1,52)  = 7.900, p  = 0.007, ω 2  = 0.11). Across experimental setups, patients had a similar path error as controls in learning trials (group: F (1,52)  = 0.630, p  = 0.431, ω 2  = 0.0). In probe trials, we found a decrease in path error by 53.6% in patients (Mean ± SEM: 278.8 ± 55.1 vs. 124.3 ± 28.1) and by 53.7% in controls (Mean ± SEM: 251.2 ± 31.9 vs. 116.2 ± 15.5; setup: F (1,27)  = 48.153, p  < 0.001, ω 2  = 0.62) in the mobile setup. Across experimental setups, patients had a similar path error as controls in probe trials (group: F (1,25)  = 0.265, p  = 0.611, ω 2  = 0.0).

The improved spatial efficiency was also reflected in lower surface coverage when multisensory input was available in the mobile VR setup. In learning trials, average surface coverage decreased by 36.9% in patients (Mean ± SEM: 38.8 ± 3.5 vs. 24.5 ± 1.8) and by 23.5% in controls (Mean ± SEM: 30.2 ± 1.9 vs. 23.1 ± 1.6; setup: F (1,28)  = 33.499, p  < 0.001, ω 2  = 0. 53). Across experimental setups, patients had a similar surface coverage as controls in learning trials (group: F (1,25)  = 4.755, p  = 0. 039, ω 2  = 0. 12). In probe trials, we found a decrease in surface coverage by 34.9% in patients (Mean ± SEM: 24.9 ± 2.8 vs. 16.2 ± 1.9) and by 18.8% in controls (Mean ± SEM: 22.9 ± 1.6 vs. 18.6 ± 1.7; setup: F (1,27)  = 26.254, p  < 0.001, ω 2  = 0. 47) in the mobile setup. Across experimental setups, patients had a similar surface coverage as controls in probe trials (group: F (1,25)  = 0.008, p  = 0.929, ω 2  = 0. 0).

In contrast to spatial memory performance and navigation strategies, we found an influence of the session order on navigation efficiency, at least for the performance in probe trials. The result indicates that with increasing experience with the task itself navigation efficiency increases (Probe trials: latency, F (1,27)  = 8.096, p  = 0.008, ω 2  = 0.20; path error, F (1,27)  = 21.206, p  < 0.001, ω 2  = 0.41; surface coverage, F (1,27)  = 12.404, p  = 0.002, ω 2  = 0.28; see Supplementary Table  4 ).

Multisensory input modulates navigation strategies in patients with hippocampal lesions

To achieve the goal of navigating to the target location, a range of different strategies were available to participants. We used the observed movement patterns, such as the shape of the path to a location as well as the rotational behavior of the navigators, to infer on the underlying navigation strategies. We extracted three parameters that reflect different strategies that participants employed to find the target in the water maze: search accuracy, landmark use, and path replication. The choice of one strategy does not preclude the use of other strategies, as participants may switch between strategies and use more than one strategy simultaneously on the way to the target location 21 , 32 .

Search accuracy describes the spatial focus of the search behavior in the water maze. It is characterized by the average distance to the final location 33 , 34 , (Fig.  4a–c ). A lower average distance reflects a preference for more intensive and focused search of the object near the final location, while higher averaged distance is found when participants are primarily searching randomly or distant from the final location (see methods).

a Schematic representation of the calculation of the average distance to the final location as a measure of search accuracy. b Multisensory input in the mobile setup increased patients’ search accuracy more than controls search accuracy in learning trials (group*setup: F (1,53)  = 4.442, p  = 0.040, ω 2  = 0.06; stat-MTRL vs. mobile-MTLR, p  = 0.004; stat-control vs. mobile-control, p  = 0.219). c Multisensory input in the mobile setup increased patients’ and controls’ search accuracy to a similar extent in probe trials (setup: F (1,28)  = 20.726, p  < 0.001, ω 2  = 0.40). d Schematic representation of the calculation of the angular velocity over first five seconds as a measure for landmark use for self-localization, and path planning. e Multisensory input in the mobile setup increased patients’ and controls’ landmarks use to a similar extent in learning trials (setup: F (1,27)  = 32.830, p  < 0.001, ω 2  = 0.52), but patients used less landmarks across setups (group: F (1,25)  = 9.206, p  = 0.006, ω 2  = 0. 23). f Multisensory input in the mobile setup increased patients’ landmarks use less than controls resulting group differences in probe trials (setup*group: F (1,53)  = 5.375, p  = 0.024, ω 2  = 0.07; stat-MTLR vs. mobile-MTLR, p  < 0.001; stat-control vs. mobile-control, p  < 0.001; stat-MTLR vs. stat- control, p  = 0.477; mobile-MTLR vs. mobile-control, p  < 0.001). g Schematic representation of the calculation of the trajectory distance using dynamic time warping for alignment of trajectories to compare as a measure for path replication of the last learning trial trajectory and each probe trial trajectory. h Multisensory input in the mobile setup increased patients’ use of path replication more than controls (setup*group: F (1,52)  = 10.723, p  = 0.002, ω 2  = 0.15; stat-MTLR vs. mobile-MTLR, p  = 0.002; stat-control vs. mobile-control, p  = 0.759; stat-MTLR vs. stat- control, p  = 0.063; mobile-MTLR vs. mobile-control, p  = 0.079). Metric data presented as boxplots with a center line as median, Tukey-style whiskers extend 1.5 times the interquartile range from 25 th and 75 th percentiles. Dots present individual datapoints. Data was analyzed with a linear mixed model. Sample size, MTRL: n  = 10, control: n  = 20; ∗  =  p  ≤ 0 .05; ∗ ∗  =  p  ≤ 0.01; ∗ ∗ ∗  =  p  ≤ 0.001.

In learning trials, we found that the availability of multisensory input in the mobile VR setup led to greater improvement in search accuracy in patients than in controls. The increase was reflected in a decrease in the average distance to the final location by 19.0% in patients (Mean ± SEM: 0.42 ± 0.01 vs. 0.34 ± 0.01) and by 7.9% in controls (Mean ± SEM: 0.38 ± 0.01 vs. 0.35 ± 0.01; group*setup: F (1,52)  = 4.456, p  = 0.040, ω 2  = 0. 06; stat-MTLR vs. mobile-MTLR, p  = 0.003; stat-control vs. mobile- control, p  = 0.163; stat-MTLR vs. stat-control, p  = 0.081; mobile-MTLR vs. mobile- control, p  = 0.566). In contrast, in probe trials, we found that the availability of multisensory input in the mobile VR setup increased search accuracy to a similar extent for both groups. The increase was reflected in a decrease in the average distance to the final location by 15.2% in patients (Mean ± SEM: 0.33 ± 0.02 vs. 0.28 ± 0.02) and by 16.7% in controls (Mean ± SEM: 0.30 ± 0.01 vs. 0.25 ± 0.02; setup: F (1,27)  = 22.429, p  < 0.001, ω 2  = 0. 42). Across experimental setups, patients had a similar search accuracy as controls in probe trials (group: F (1,25)  = 1.644, p  = 0.212, ω 2  = 0. 02).

The use of landmarks is the most efficient strategic behavior in an allocentric spatial navigation task such as the MWM 15 , 19 . Early incorporation of information from the surroundings, such as landmarks, accelerates self-localization, localization of the target location, and eventually the calculation of the optimal path. A measure of the estimated use of landmarks for path planning is the integrated absolute angular velocity at the start of a trial 35 , 36 , (Fig.  4d–f ). The measure describes the extent of head movements, and a higher value refers to more intense use of the surrounding landmarks (see methods).

In learning trials, we found that the availability of multisensory input in the mobile VR setup increased the use of landmarks to a similar extent in patients and controls. The increase in landmark use was reflected in an increase in average angular velocity by 31.1% in patients (Mean ± SEM: 0.0060 ± 0.0006 vs. 0.0081 ± 0.0005) and by 36.8% in controls (Mean ± SEM: 0.0076 ± 0.0004 vs. 0.0105 ± 0.0005; setup: F (1,27)  = 32.830, p  < 0.001, ω 2  = 0. 52). Across experimental setups, patients had a lower angular velocity than controls in learning trials (group: F (1,25)  = 9.206, p  = 0.006, ω 2  = 0. 23). In contrast, in probe trials, we found that the availability of multisensory input in the mobile VR setup increased the use of landmarks significantly less in patients than in controls. This was reflected in an increase in angular velocity by 122.2% in patients (Mean ± SEM: 0.0036 ± 0.0005 vs. 0.0080 ± 0.0004) and by 161.0% in controls (Mean ± SEM: 0.0041 ± 0.0003 vs. 0.0107 ± 0.0005; setup*group: F (1,52)  = 5.375, p  = 0.024, ω 2  = 0.07; stat-MTLR vs. mobile-MTLR, p  < 0.001; stat-control vs. mobile-control, p  < 0.001; stat-MTLR vs. stat- control, p  = 0.477; mobile-MTLR vs. mobile-control, p  < 0.001).

Path replication is a strategic behavior based on route-based learning. It is realized by repeatedly navigating to the location of an object from a fixed starting location in learning trials. The use of repeated path sequences to reach the final location relies on egocentric representations, even when approaching the target location from a new location 18 , 21 , 32 , 37 . The extent of repetition is reflected in the distance between the aligned trajectories of the last learning trial and the trajectories for each probe trial (Fig.  4g, h ). A smaller distance between the trajectories reflects a stronger repetition of the path, while large distances represent dissimilar trajectories (see methods).

We found that the availability of multisensory input in the mobile VR setup led to replication of path sequences significantly more in patients than in controls. The increase in path replication was reflected by a decrease in trajectory distance by 23.3% in patients (Mean ± SEM: 0.30 ± 0.01 vs. 0.23 ± 0.02) and by 0.0% in controls (Mean ± SEM: 0.26 ± 0.01 vs. 0.26 ± 0.01; setup*group: F (1,52)  = 10.723, p  = 0.002, ω 2  = 0.15; stat-MTLR vs. stat-control, p  = 0.063; mobile-MTLR vs. mobile-control, p  = 0.079; stat-MTLR vs. mobile-MTLR, p  = 0.002; stat-control vs. mobile-control, p  = 0.759).

We investigated effects of multisensory input on memory-guided spatial navigation in humans with and without hippocampal lesions. To this end, we used a virtual version of the MWM, a classic paradigm for testing memory-guided spatial navigation. The task was presented in either a stationary desktop setup with mainly visual input or a mobile immersive VR setup with multisensory input. Our results show that multisensory input modulated distinct aspects of memory-guided spatial navigation including spatial memory performance, navigation efficiency, and navigation strategies. Both, patients with hippocampal lesions and healthy control participants showed overall improvements in memory-guided spatial navigation when multisensory input was available. Remarkably, spatial memory performance improved more in patients than in control participants. In addition, the availability of multisensory information affected navigation strategies of patients with hippocampal lesions and control participants differently: whereas control participants employed more spatial landmarks to navigate to remembered locations, patients showed stronger replication of path sequences when they navigated freely compared to when they performed the same task in a stationary setup. Our results show that rearranged processing of multisensory input can efficiently compensate for hippocampal damage and should be taken into consideration when interpreting navigational behavior in human patients.

We observed improvements in spatial memory performance and navigation efficiency in humans with and without hippocampal lesions depending on the availability of multisensory input. Our results emphasize that multisensory input has direct implications for memory-guided spatial navigation performance. The observed behavioral changes can be explained by a modulation of neural activity across multiple brain areas including extrahippocampal brain regions. Indeed, functional magnetic resonance imaging studies have shown that the hippocampus and adjacent entorhinal and parahippocampal cortices are critical for processing spatial information during navigation 38 , 39 . However, these brain regions are part of a broader navigational network that extends beyond the medial-temporal lobe 1 , 13 . Within this network the hippocampus and adjacent structures interact and share computations for ego- and allocentric spatial representations with the striatum and neocortical brain areas such as the posterior parietal cortex and the retrosplenial cortex 1 , 40 , 41 . While the striatum contributes to stimulus-response learning, the retrosplenial cortex and the posterior parietal cortex integrate spatial information derived from head and body movements with visual spatial information based on landmarks which facilitates the localization of the self and familiar locations 42 , 43 , 44 .

When participants navigate in the real world or in immersive VR, the visual system, vestibular organ, and proprioception relay congruent and complementary sensory information to the retrosplenial cortex and the posterior parietal cortex 45 . This partially overlapping information from multimodal sources is then processed with low computational noise, which promotes the formation of robust spatial memory representations 46 . In contrast, when participants navigate virtual space projected on a desktop screen, body-based sensory input indicates that participants are stationary. These signals are at odds with the visual flow that simulates the experience of movement. As a result, different brain areas need to reconcile conflicting sensory information, leading to a greater cognitive demand for the generation of coherent spatial representations. These contradicting representations may alter the strategies for solving the navigation task and underlying neural processes 24 , 25 , further disadvantaging participants in the stationary desktop setup compared to the mobile setup. Consequently, congruent sensory inputs to the posterior parietal cortex and retrosplenial cortex may have contributed to the improvements in performance observed in our study when body motion of participants was unrestricted and matched visual input.

Our study suggests that memory-guided spatial navigation in situations without multisensory input is more dependent on the integrity of the medial temporal lobe. The medial temporal lobe processes spatial features from visual flow independent of the viewpoint. For example, grid cells in entorhinal cortex extract the metric features of space, and the parahippocampal cortex processes information about landmarks even when self-motion is restricted 1 , 47 , 48 , 49 , 50 . This provides a contextual framework for spatial relations and enables processing of egocentric and allocentric representations in the hippocampus 40 , 51 , 52 . However, without complementary computations in extrahippocampal brain regions, the accuracy of the spatial representation relies on the accurate encoding of spatial information by the hippocampus. In this case, the influence of a hippocampal lesion becomes particularly evident. In humans, the hippocampal function has been shown to be lateralized, with the right hippocampus linked more to navigation-related functions compared to the left hippocampus 53 , 54 . The patients in our study had lesions in the right medial temporal lobe and they exclusively relied on the left medial temporal lobe for hippocampal computations. This resulted in discernible deficits in memory-guided spatial navigation especially in the stationary setup, with the absence of congruent body-based spatial representations hosted by extrahippocampal brain areas.

Our results further show that the availability of multisensory input compensates for deficits in memory-guided spatial navigation due to a hippocampal lesion. In the hippocampus, unique combinations of place cells encode specific locations in relation to environmental cues 12 , 15 , 55 . However, even when hippocampal place cells are dysfunctional or absent, navigation to a target location in relation to an allocentric frame of reference is possible 30 , 56 . One explanation is that hippocampal dysfunction is compensated for by increased computations in neocortical brain regions, evidenced by an altered frontoparietal activation pattern correlating with behavior 57 . In the broader network for spatial navigation, the medial temporal lobe and the parietal areas are likely to communicate via the retrosplenial cortex 58 . It is thus plausible that parietal and retrosplenial areas are more heavily recruited in absence of right hippocampal input, when the participants perform navigation tasks. Such a shift in computations for mental representations may subsequently translate into behavioral changes. Specifically, redistribution of computation from the hippocampus to the retrosplenial cortex and posterior parietal cortex can alter the relative proportion of allocentric versus egocentric spatial representations. In the joint integrative processing of spatial information, it would be adaptive to put more weight onto the intact parietal or retrosplenial representations that are anchored to egocentric reference frames. On the behavioral level, this would be reflected in the change in preferred navigation strategies. Indeed, we observed that patients with hippocampal lesions showed an increased use of egocentric navigation strategies and used landmark information less intensively in response to multisensory input compared to controls.

It should be noted, however, that the modulation in spatial navigation observed in our patients cannot be attributed exclusively to the use of egocentric representations. Performance above chance level in our version of the water maze requires allocentric representations, specifically in the probe trials with varying starting locations. In this scenario, a complete reliance on egocentric strategies such as path replication would misguide the participant, as they are required to approach the target location from different viewpoints. One explanation for the above chance performance in these trials is that computations for forming the allocentric representations overlap in both hippocampus and neocortex 1 , 13 . With allocentric representations available, egocentric representations can be embedded in the correct environmental context, translating otherwise misleading egocentric coordinates into an allocentric frame of reference.

Our systematic comparison of memory-guided navigation in humans in a stationary and a mobile setup can explain some of the controversial results in previous navigation studies. In prior research, it has been observed that patients with hippocampal lesions showed profound impairments in spatial memory performance in stationary virtual navigation tasks 19 . Patients with acute or chronic hippocampal lesions were affected, and the effect was particularly pronounced when the lesion was located in the right medial temporal lobe 20 , 59 , 60 . In contrast to observations in stationary navigation tasks, patients with unilateral hippocampal lesions were able to solve a physical analog of the water maze as efficiently as healthy controls 30 , 61 , and even patients with bilateral hippocampal lesions navigated a physical or immersive VR version of the water maze better than chance 30 , 56 , 62 . Our results indicate that these conflicting findings can—at least partially—be attributed to the relevant difference in task design, namely the degree of mobility and the availability of multisensory input.

The importance of mobility for studying navigation is further highlighted by the fact that most of our understanding on the neural underpinnings of memory-guided spatial navigation is derived from behavioral experiments in freely moving animals. Due to interspecies differences, results from rodent studies cannot be readily translated to humans. However, in humans, navigation is often assessed in an immobile supine, sitting, or standing position in electrophysiological, imaging, and lesion studies 63 , 64 , 65 . In contrast, higher degrees of mobility in immersive VR environments allow for a more ecological comparison of knowledge about navigation between animals and humans. Combined with advances in mobile brain imaging technology, such as high-density mobile electroencephalography, optically pumped magnetoencephalography, and intracranial leads, immersive VR environments provide the opportunity to study neural substrates during full-body movement 66 , 67 , 68 , 69 , 70 . These methodological approaches can help identify shifts in neural substrates in response to changing behavioral demands on spatial navigation in future studies.

Our study highlights the importance of considering contextual factors as modulators of spatial navigation. We observed that multisensory input has a profound impact on memory-guided spatial navigation. Behavioral patterns may change significantly in response to contextual factors. Since unrestricted movement is a key feature of natural spatial navigation, behavioral observations in non-mobile navigation studies should be interpreted carefully and mobility should be allowed whenever possible.

Beyond a better understanding of spatial navigation, humans with deficits in spatial navigation especially benefit from improved ecological validity in studies. This includes patients with neurodegenerative diseases, but also patients with acute lesions to the navigational network, e.g., due to stroke or inflammatory brain disorders. Deficits of navigational abilities in these patients could be treated by rehabilitation schemes that promote compensatory mechanisms in the navigational network.

In conclusion, the behavioral data from our experiment support the assumption of an extended large-scale navigation network in which brain regions continuously share the processing of egocentric and allocentric spatial representations rather than performing temporally and spatially separated computations in distinct neural substrates 1 , 13 . Complementary and redundant computations across brain regions allow for a flexible shift of processing according to current behavioral demands and reliability of spatial representations. Our results furthermore highlight the importance of considering contextual factors such as the availability of multisensory input in studies on memory-guided spatial navigation in patients with hippocampal damage and in healthy participants.

Participants

In total, thirty-four participants took part in the experiment and thirty participants were included in the final dataset of our study (18 female, 12 male; Table  1 ). Eleven patients were recruited through our Department of Neurology who had undergone unilateral partial resection of the right medial temporal lobe (MTLR), including the hippocampus, due to hippocampal sclerosis and intractable epilepsy ( n  = 7) or due to removal of a benign tumor ( n  = 4) (Fig.  5 , Table  2 ). The other inclusion criteria for patients were as follows: Age 18–65 years, fluent German (at least C1 level), postoperative neurological examination was normal, no other neuropsychiatric or severe internal diseases were reported, vision and hearing were normal or corrected to normal, no subjective memory complaints and navigation deficits in daily life were reported, and patients could be fully reintegrated into their personal and professional lives after surgery. Another requirement for inclusion in the study was a postoperative period of at least 6 months before the test to ensure sufficient recovery time after surgery. Each patient was matched with two healthy control participants in terms of gender, age, and education level. The control subjects were recruited via online advertising. Participants in the final data set were aged between 22 and 61 years, and four patients were taking anticonvulsant medication at the time of the study. Clinical cognitive assessment was not considered in the recruitment of patients, as patients with unilateral lesions are more likely to have subtle memory deficits that are not usually detected in routine examinations 71 . One patient and three control participants were later excluded due to cyber sickness or an additional neuropsychiatric disorder that was not known at the time of the experiment. All participants provided written informed consent in accordance with the Declaration of Helsinki and all procedures were approved by the local ethics committee of Charité-Universitätsmedizin Berlin. All ethical regulations relevant to human research participants were followed.

Postoperative coronal T1 MRI images of the brain show the unilateral lesion of the medial temporal lobe including the right-sided hippocampus, while the left-sided hippocampus is intact.

Lesion evaluation

All patients except No. 5 and No. 9 participated in previous studies where lesion size was analyzed 72 , 73 . Nos. 5th and 9th lesions were additionally analyzed using MRI scans from clinical routine. Briefly, 47 coronal T1 sections of the whole brain with an individual thickness of 4 mm were used to determine individual lesion size. The extent of each lesion was determined from rostral to caudal using previously proposed landmarks 74 , 75 , 76 , 77 .

Experimental setup

To investigate the influence of multisensory input on memory-guided spatial navigation, we tested spatial memory and navigation behavior in a virtual environment presented either on a screen on which participants navigated with a joystick (stationary) or in an immersive VR setup in which participants moved freely (mobile) (Fig.  1a, b ).

The duration of the entire experiment varied between four and six hours, including technical preparations and breaks. We performed all experiments at the Berlin Mobile Brain/Body Imaging Labs (BeMoBIL) at the Technische Universität Berlin.

Participants performed the task equipped with a fully mobile EEG system (Supplementary Methods  2 ), the data from which will be reported in detail in a follow-up study with the focus on the electrophysiological dynamics during spatial navigation. EEG data will be analyzed to confirm or reject the hypotheses about brain dynamics raised in the current study.

Virtual navigation task

To investigate memory-guided spatial navigation, we used a modified virtual version of the MWM-task developed in Unity 3D (v.2018.4.13f1), 14 , 15 , 78 . The virtual environment consisted of an open, circular arena surrounded by environmental cues (Fig.  1c, d ). The arena had a radius of 3.8 (virtual) meters and was bounded by a 1.7 (virtual) meter high wall. The ground inside the arena was covered with a half-transparent fog model. A skybox with clouds was rendered in the background and the arena was located in the valley of a hilly terrain. Three different buildings were placed in a triangular formation in the hilly landscape (Medieval house 3D, PBR medieval houses pack, Church model, Medieval village environment, Medieval castle pack available in Unity 3D store; Unity’s standard Unity Asset Store End User License Agreement (EULA), extension asset). To avoid carryover effects between the stationary desktop setup and the mobile VR setup, we created two different versions of the virtual environment (Fig.  1c ). The scene versions and presentation order for the stationary setup and the mobile setup were matched between participants.

Behavioral testing

Six experimental blocks were presented in each of the stationary desktop and mobile VR setups (Fig.  1d , Supplementary Table  1 ). A block was defined by a set of spatial parameters, namely the start location and the target location. First, the six start locations were located on each end of the radial axes, equally dividing the circle into six areas (at 0, 60, 120, 180, 240, and 300 degrees). For each start location, the corresponding target location was located on one of the center axes of the four quadrants defined with respect to the start location (relative angles ± 45 degrees or ±135 degrees). The distance of the target location from the center was randomly sampled from a uniform distribution over the interval of [0.2, 0.8] × arena radius (3.8 (virtual) meters). The six sets of block-specific spatial parameters were generated once and used for all participants and both sessions (Supplementary Table  1 ). However, the order of the blocks was randomly permuted for each setup. At the target location a randomly selected object model (Toys Pack, Lowpoly Flowers, 3D Cute Toy Models available in Unity 3D store; Unity’s standard Unity Asset Store End User License Agreement (EULA), extension asset) was used per block.

Each block started with three learning trials, followed by four probe trials. A disorientation task was inserted between every consecutive pair of trials. In learning trials, participants searched for the hidden target object in the arena. The object gradually appeared when approached (<1.2 (virtual) meters) and was registered as found when the participant was closer than 0.8 (virtual) meters. The target object remained visible for a maximum of 20 s, and the participant was instructed to remember its location. This phase could optionally be ended earlier by pressing a key.

In probe trials, participants were asked to navigate back to the remembered target location. While the target location remained fixed, the start location varied between the four probe trials. The start locations were defined as rotations of the origin—the start location used during learning—around the center by 0, 90, 180, and 270 degrees. The four rotations were presented in a randomly permuted order within a block. In these trials, the target object stayed invisible, and the participants completed the task by pressing the key after having positioned themselves at the remembered location of the target.

A disorientation task was inserted between all pairs of consecutive trials or after termination of a break between blocks in both the stationary and mobile sessions. This was to prevent participants from using a simplistic strategy of immediately backtracking the learned trajectory from the previous trial. In the disorientation task, all visual features that could be used as a spatial cue were hidden, including the skybox. Participants were first asked to navigate to a waypoint—a blue sphere—at the center of the arena. Then a white sphere appeared in the viewing direction, which guided the participant to turn their body following a sequence of three rotations. The rotation sequence was randomized between right-left-right and left-right-left. After following the sequence of rotation, they were asked to walk straight to the starting location of the next trial indicated by a waypoint. Only then the sky and other spatially relevant features in the virtual environment were revealed again and the next trial started. The reasoning behind this manipulation was that the representation of the location of oneself formed in a trial should be reset at the beginning of the next one. As it is physically challenging to teleport participants in real world, we have rotated the virtual environment and masked the potential dissonance with the disorientation task.

Prior behavioral testing, all participants became acquainted to the task requirements such as the mode of control by performing a baseline and a practice block before the start of the first experimental block in each setup. A baseline block consisted of a phase of 30 s where they stood still looking in a specific direction from within the arena, followed by a navigation task where they followed 36 waypoints appearing in the arena one after another. Per setup, the baseline block was presented three times: before the first experimental block, after the third block, and at the end of the sixth (last) block. The practice block was presented only once at the beginning of each setup by default and then demonstrated one learning trial and one probe trial with a disorientation task in between.

Technical equipment: stationary desktop setup

In the stationary setup, the virtual environment was presented with a first-person view on a wall-mounted screen (43 inches, 3840 × 2160) in the same room as the mobile VR setup. Participants viewed the screen while standing approximately 1.2 meters away and simulated movement using a joystick (Speedlink Dark Tornado) placed on a desk in front of them. The heights of both the screen and the desk were adjusted according to the height of the participant. To navigate in the virtual environment, participants rotated their perspective around the up-down axis (yaw) by tilting the joystick to the left or to the right. Likewise, forward and backward translation was controlled by tilting the joystick forward or backward. The speed of translation was 1.4 virtual meters per second. The rotation speed was 50 degrees per second. The time series of positions and orientation data of the virtual camera was sampled at 60 Hz (refresh rate of the display) and streamed to the Lab Streaming Layer 79 . Participants pressed a red button on the joystick to respond or to terminate breaks between blocks.

Technical equipment: mobile VR setup

In the immersive mobile VR setup, a virtual environment was presented to the participants using a head-mounted immersive VR display (HTC Vive Pro, 110 degrees field of view). To enable wireless navigation within the room, a wearable gaming PC (Zotac: PC Partner Limited), powered by portable batteries, was used to generate the graphical input to the HMD. The time series of positions and orientation data of the HMD were sampled at 90 Hz (refresh rate of the display) and were streamed via Wi-Fi to the Lab Streaming Layer on the recording PC 79 . The navigable area in the room was ~15 × 9 meters. However, participants were instructed to always stay within the boundary of the virtual arena (a walled circle with 3.8 meters radius). During the task, there were no external cues (sound or air flow) that may have informed participants of their position in the room. While performing the tasks, participants held an HTC Vive controller and pressed the trigger key to respond or to terminate breaks between blocks.

Data analysis

We recorded the participant’s position in the virtual environment as x, y coordinates in a Cartesian coordinate system along with a time stamp and rotations as quaternions. Yaw angles were computed by converting quaternions to Euler angles and the channel preprocessed with a 6 Hz low pass filter to capture only those motions that had a relevant time scale. We pre-processed the navigation data in MathWorks® Matlab (version 2021a). Using the position and yaw data, we computed different variables to capture distinct aspects of memory-guided spatial navigation. Here, we focused on spatial memory performance, navigation efficiency, and navigation strategies.

First, we assessed spatial memory performance by determining how well participants could remember the target locations in the test trials and how accurate and consistent the underlying spatial memory representations were. To this end, we computed the memory score and the scatter of final locations.

Memory score was chosen as a measure for spatial memory. For each trial, we calculated the Euclidean distance between the target location and the final location. We compared this value to a reference distribution obtained by calculating the Euclidean distance between each target location and 1000 randomly selected locations in the arena. The memory score corresponds to the percent rank of the distribution, i.e., the proportion of randomly selected locations that were farther from the target location than the final location. Thus, the memory score ranges from zero to 100%, with 100% representing a perfect recall rate and 50% representing randomness. A value between 50% and 0 indicates a systematic bias in the false direction 28 , 29 .

Scatter of final locations was chosen as a measure for spatial precision which also depends on hippocampal integrity 30 , 31 . Regardless of the distance to the actual target location, precision indicates how consistent responses were per target location. The scatter of participants’ responses was calculated as the average distance between each of the four final locations per object location. A smaller scatter meant higher spatial precision (Fig.  2c ).

Second, we evaluated temporal and spatial navigation efficiency. The temporal efficiency describes how quickly the final location was reached, while the spatial efficiency describes how directly the final location was reached.

Latency to final location was determined as measure for temporal efficiency. We calculated the latency to final location by subtracting the time of the trial onset from the time of the trial offset, in seconds.

Path error was computed as first measure for spatial efficiency. Path error describes the directness of the participant’s path to the final location. Here, we calculated the length of the participant’s path and the ideal path length to the final location (ideal path is defined as the straight line from the start location to the final location, and its length corresponds to the Euclidean distance between two locations). We subtracted the ideal path length from the measured path length to obtain the excess path length. The excess path length was divided by the ideal path length and finally multiplied by 100, to yield the path error. The path error ranges from zero to infinity, with higher values representing less direct paths.

Surface coverage was used as second measure for spatial efficiency. Surface coverage refers to the maximum proportion of the arena surface visited by participants and provides an estimate of detours and the amount of target oriented spatial search during navigation. We calculated the difference between the minimum and maximum x and y coordinates, respectively, and determined the area of an ellipse to obtain an estimate of the area covered by the participants. We divided this value by the actual area of the arena to obtain the proportional amount of area covered. Higher values for covered area represent low spatial efficiency.

Third, we assessed the navigation strategies underlying participants’ performance. To this end, we used the observed movement patterns, such as the shape of the path to a location as well as the rotational behavior of the navigators, to infer the underlying navigation strategies. We distinguished three different parameter that reflect strategies that participants employed to find the target in the water maze: search accuracy, landmark use, and path replication. The choice of one strategy does not preclude the use of other strategies, as participants may switch between strategies and use more than one strategy simultaneously on the way to the target location 21 , 32 .

Average distance to final location was computed as a measure for search accuracy, which describes the preferred spatial focus of the search behavior in the water maze. The focus of a search can be at the start location, at the final location, in the middle of the arena or randomly distributed in the arena. The focused location then has a higher-than-average number of coordinate points. A lower average distance reflects a preference for a more intense and focused search for the object near the final location, while a higher average distance is found when participants search mainly randomly or far away from the final location 33 , 34 .

Initial angular velocity was evaluated as measure for landmark use. Landmark use describes the degree of visual exploration of the environment containing landmarks. The use of landmarks is the most purposeful strategic behavior in an allocentric spatial navigation task such as the water maze 15 , 19 . Efficient acquisition of information from the environment, such as landmarks, at the beginning of navigation accelerates self-localization, localization of the target location, and finally computation of the optimal path. We used integrated absolute angular velocity (idPhi) to quantify the extent to which participants used information from the environment. idPhi is derived from the heading data by unwrapping the yaw angles and taking the derivatives to calculate angular velocity. The instantaneous angular velocity values were averaged over the time window of interest to represent how much the participant had turned their head laterally. We chose the first five seconds as the time window of interest, as we were particularly interested in the exploration of the environment at the beginning of each trial. idPhi is commonly used as an index of vicarious trial and-error behavior and is known to be affected by impairments of the medial temporal lobe 35 , 36 , 80 .

Trajectory distance was determined as measure for path replication. Path replication describes the repetition of previously learned paths or path elements. This behavior requires route-based learning, which is realized by repeatedly navigating to the location of an object from a fixed location in learning trials. The use of repeated path sequences to reach the final location relies on egocentric representations, even when approaching the target location from a new start location 18 , 21 , 32 , 37 . The extent of repetition is reflected in the distance between the aligned trajectories of the last learning trial and the trajectories for each probe trial. We aligned the trajectory of the final learning trial with the trajectory of each probe trial by first rotating both trajectories around the center to obtain the same start coordinates, and then aligning the trajectories regardless of the actual path length using Matlab’s dynamic time warping function dtw . To normalize the trajectory distance considering the number of data points, the dtw-distance was divided by the smallest number of matrix cells to be visited 81 . A smaller distance between matching trajectories reflects a greater repetition of the path or its elements, while large distances represent dissimilar trajectories.

Statistics and reproducibility

We performed the statistical analysis in R (v. 3.5). To determine whether our behavioral data met the assumption of a normal distribution, we applied the Shapiro-Wilk test. If the assumption of normal distribution was violated, we assessed the skewness and kurtosis of the data and applied a log transformation if the skewness was less than −2 or greater than 2 or the kurtosis was <−7 or >7, respectively.

Because our dataset consisted of consecutive measurements in two different experimental setups, we analyzed our data using a linear mixed model for two-sided testing and designed with the R package lme4 (v.1.1-35), 82 , 83 . Fixed effects were group (between participants factor with two levels: MTLR and control) and setup (within-participants factor with two levels: stationary and mobile), and model covariates included session order, participant sex, age, and years of education, and random effects included participant ID to account for interindividual differences. The model was estimated using the restricted maximum likelihood method and degrees of freedom were calculated using the Satterthwaite method 84 . In case the main analysis revealed a significant interaction effect, a post-hoc test was performed using the R package emmeans with the Holm-Bonferroni correction for multiple comparisons to prevent an increase in type-I-errors (v.1.8.5), 85 . The R package effectsize was used to calculate effect sizes as Omega squared (ω 2 ) (v.0.8.5).

To ensure comparable group characteristics with respect to sex age, and years of education, we used either the χ2-independence test for nominal variables or the nonparametric Kruskal-Wallis test for metric variables. For all statistical tests applied, we set the significance level to the conventional level of 0.05.

The sample size for all statistical tests was as follows: MTLR, n  = 10; control, n  = 20. We provide two tables of results for learning trials and probe trials respectively (Supplementary Tables  2 , 3 ). Data are presented as mean ± s.e.m. and 95% confidence interval. For the effect of the session order on the experimental variables, another table of results is provided with test statistics, p value, and effect size (supplementary tables  4 ). The data are presented as box-and-whisker plots with a center line representing the median and with individual data points overlaid to show the full data distribution.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this study are available at the Open Science Framework (osf) at: https://osf.io/u47mj/ , (unique identifier: https://doi.org/10.17605/OSF.IO/U47MJ ).

Code availability

Our matlab- and R-functions are available at the Open Science Framework (osf) at: https://osf.io/u47mj/ , (unique identifier: https://doi.org/10.17605/OSF.IO/U47MJ ). The software for the virtual water maze and the acquisition of trajectories is available upon reasonable request from the corresponding author.

Ekstrom, A. D., Huffman, D. J. & Starrett, M. Interacting networks of brain regions underlie human spatial navigation: a review and novel synthesis of the literature. J. Neurophysiol. 118 , 3328–3344 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Ekstrom, A. D. & Hill, P. F. Spatial navigation and memory: a review of the similarities and differences relevant to brain models and age. Neuron 111 , 1037–1049 (2023).

Article   CAS   PubMed   Google Scholar  

Epstein, R. A., Patai, E. Z., Julian, J. B. & Spiers, H. J. The cognitive map in humans: spatial navigation and beyond. Nat. Neurosci. 20 , 1504–1513 (2017).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Anastasiou, C., Baumann, O. & Yamamoto, N. Does path integration contribute to human navigation in large-scale space? Psychon. Bull. Rev . https://doi.org/10.3758/s13423-022-02216-8 (2022).

Brandt, T. et al. Vestibular loss causes hippocampal atrophy and impaired spatial memory in humans. Brain J. Neurol. 128 , 2732–2741 (2005).

Article   Google Scholar  

Chrastil, E. R., Nicora, G. L. & Huang, A. Vision and proprioception make equal contributions to path integration in a novel homing task. Cognition 192 , 103998 (2019).

Article   PubMed   Google Scholar  

Ottink, L. et al. Cognitive map formation supported by auditory, haptic, and multimodal information in persons with blindness. Neurosci. Biobehav. Rev. 140 , 104797 (2022).

Smith, P. F. et al. The effects of vestibular lesions on hippocampal function in rats. Prog. Neurobiol. 75 , 391–405 (2005).

Chrastil, E. R. & Warren, W. H. Active and passive spatial learning in human navigation: acquisition of graph knowledge. J. Exp. Psychol. Learn. Mem. Cogn. 41 , 1162–1178 (2015).

Jabbari, Y., Kenney, D. M., von Mohrenschildt, M. & Shedden, J. M. Vestibular cues improve landmark-based route navigation: a simulated driving study. Mem. Cognit. 49 , 1633–1644 (2021).

Nyberg, N., Duvelle, É., Barry, C. & Spiers, H. J. Spatial goal coding in the hippocampal formation. Neuron 110 , 394–422 (2022).

O’Keefe, J., Burgess, N., Donnett, J. G., Jeffery, K. J. & Maguire, E. A. Place cells, navigational accuracy, and the human hippocampus. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 353 , 1333–1340 (1998).

Ekstrom, A. D., Arnold, A. E. G. F. & Iaria, G. A critical review of the allocentric spatial representation and its neural underpinnings: toward a network-based perspective. Front. Hum. Neurosci. 8 , 803 (2014).

Garthe, A. & Kempermann, G. An old test for new neurons: refining the Morris water maze to study the functional relevance of adult hippocampal neurogenesis. Front. Neurosci. 7 , 63 (2013).

Morris, R. G., Garrud, P., Rawlins, J. N. & O’Keefe, J. Place navigation impaired in rats with hippocampal lesions. Nature 297 , 681–683 (1982).

Gramann, K. Embodiment of spatial reference frames and individual differences in reference frame proclivity. Spat. Cogn. Comput. 13 , 1–25 (2013).

Makeig, S., Gramann, K., Jung, T.-P., Sejnowski, T. J. & Poizner, H. Linking brain, mind and behavior. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 73 , 95–100 (2009).

Google Scholar  

Amlerova, J. et al. Risk factors for spatial memory impairment in patients with temporal lobe epilepsy. Epilepsy Behav. EB 26 , 57–60 (2013).

Astur, R. S., Taylor, L. B., Mamelak, A. N., Philpott, L. & Sutherland, R. J. Humans with hippocampus damage display severe spatial memory impairments in a virtual Morris water task. Behav. Brain Res. 132 , 77–84 (2002).

Bartsch, T. et al. Focal lesions of human hippocampal CA1 neurons in transient global amnesia impair place memory. Science 328 , 1412–1415 (2010).

Bohbot, V. D., Lerch, J., Thorndycraft, B., Iaria, G. & Zijdenbos, A. P. Gray matter differences correlate with spontaneous strategies in a human virtual navigation task. J. Neurosci. Off. J. Soc. Neurosci. 27 , 10078–10083 (2007).

Article   CAS   Google Scholar  

Jonson, M., Avramescu, S., Chen, D. & Alam, F. The role of virtual reality in screening, diagnosing, and rehabilitating spatial memory deficits. Front. Hum. Neurosci. 15 , 628818 (2021).

Reggente, N. et al. Enhancing the ecological validity of fMRI memory research using virtual reality. Front. Neurosci. 12 , 408 (2018).

Taube, J. S., Valerio, S. & Yoder, R. M. Is navigation in virtual reality with FMRI really navigation? J. Cogn. Neurosci 25 , 1008–1019 (2013).

Diersch, N. & Wolbers, T. The potential of virtual reality for spatial navigation research across the adult lifespan. J. Exp. Biol. 222 , jeb187252 (2019).

Jeung, S., Hilton, C., Berg, T., Gehrke, L. & Gramann, K. Virtual Reality for Spatial Navigation. Curr. Top. Behav. Neurosci . https://doi.org/10.1007/7854_2022_403 (2022).

Lin, M.-H., Liran, O., Bauer, N. & Baker, T. E. Scalp recorded theta activity is modulated by reward, direction, and speed during virtual navigation in freely moving humans. Sci. Rep. 12 , 2041 (2022).

Bellmund, J. L. S. et al. Deforming the metric of cognitive maps distorts memory. Nat. Hum. Behav. 4 , 177–188 (2020).

Jacobs, J. et al. Direct electrical stimulation of the human entorhinal region and hippocampus impairs memory. Neuron 92 , 983–990 (2016).

Kolarik, B. S., Baer, T., Shahlaie, K., Yonelinas, A. P. & Ekstrom, A. D. Close but no cigar: spatial precision deficits following medial temporal lobe lesions provide novel insight into theoretical models of navigation and memory. Hippocampus 28 , 31–41 (2018).

Yonelinas, A. P. The hippocampus supports high-resolution binding in the service of perception, working memory and long-term memory. Behav. Brain Res. 254 , 34–44 (2013).

Iglói, K., Zaoui, M., Berthoz, A. & Rondi-Reig, L. Sequential egocentric strategy is acquired as early as allocentric strategy: parallel acquisition of these two navigation strategies. Hippocampus 19 , 1199–1211 (2009).

Gallagher, M., Burwell, R. & Burchinal, M. Severity of spatial learning impairment in aging: development of a learning index for performance in the Morris water maze. Behav. Neurosci. 107 , 618–626 (1993).

Maei, H. R., Zaslavsky, K., Teixeira, C. M. & Frankland, P. W. What is the most sensitive measure of water maze probe test performance? Front. Integr. Neurosci. 3 , 4 (2009).

Papale, A. E., Zielinski, M. C., Frank, L., Jadhav, S. P. & Redish, A. D. Interplay between hippocampal sharp wave ripple events and vicarious trial and error behaviors in decision making. Neuron 92 , 975–982 (2016).

Schmidt, B., Papale, A., Redish, A. D. & Markus, E. J. Conflict between place and response navigation strategies: effects on vicarious trial and error (VTE) behaviors. Learn. Mem. 20 , 130–138 (2013).

Barkas, L. J., Henderson, J. L., Hamilton, D. A., Redhead, E. S. & Gray, W. P. Selective temporal resections and spatial memory impairment: cue dependent lateralization effects. Behav. Brain Res. 208 , 535–544 (2010).

Brown, T. I. et al. Prospective representation of navigational goals in the human hippocampus. Science 352 , 1323–1326 (2016).

Fernandez, C., Jiang, J., Wang, S.-F., Choi, H. L. & Wagner, A. D. Representational integration and differentiation in the human hippocampus following goal-directed navigation. eLife 12 , e80281 (2023).

Alexander, A. S. et al. Neurophysiological coding of space and time in the hippocampus, entorhinal cortex, and retrosplenial cortex. Brain Neurosci. Adv. 4 , 2398212820972871 (2020).

Woolley, D. G. et al. Homologous involvement of striatum and prefrontal cortex in rodent and human water maze learning. Proc. Natl. Acad. Sci. USA 110 , 3131–3136 (2013).

Alexander, A. S. et al. Egocentric boundary vector tuning of the retrosplenial cortex. Sci. Adv. 6 , eaaz2322 (2020).

Baumann, O. & Mattingley, J. B. Extrahippocampal contributions to spatial navigation in humans: A review of the neuroimaging evidence. Hippocampus 31 , 640–657 (2021).

Gahnstrom, C. J. & Spiers, H. J. Striatal and hippocampal contributions to flexible navigation in rats and humans. Brain Neurosci. Adv. 4 , 2398212820979772 (2020).

Clark, B. J., Simmons, C. M., Berkowitz, L. E. & Wilber, A. A. The retrosplenial-parietal network and reference frame coordination for spatial navigation. Behav. Neurosci. 132 , 416–429 (2018).

Faisal, A. A., Selen, L. P. J. & Wolpert, D. M. Noise in the nervous system. Nat. Rev. Neurosci. 9 , 292–303 (2008).

Ekstrom, A. D. et al. Cellular networks underlying human spatial navigation. Nature 425 , 184–188 (2003).

Nau, M., Navarro Schröder, T., Bellmund, J. L. S. & Doeller, C. F. Hexadirectional coding of visual space in human entorhinal cortex. Nat. Neurosci. 21 , 188–190 (2018).

Luzzi, S., Pucci, E., Di Bella, P. & Piccirilli, M. Topographical disorientation consequent to amnesia of spatial location in a patient with right parahippocampal damage. Cortex J. Devoted Study Nerv. Syst. Behav. 36 , 427–434 (2000).

van Ekert, J., Wegman, J., Jansen, C., Takashima, A. & Janzen, G. The dynamics of memory consolidation of landmarks. Hippocampus 27 , 393–404 (2017).

Buzsáki, G. & Moser, E. I. Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nat. Neurosci. 16 , 130–138 (2013).

McNaughton, B. L., Battaglia, F. P., Jensen, O., Moser, E. I. & Moser, M.-B. Path integration and the neural basis of the ‘cognitive map’. Nat. Rev. Neurosci. 7 , 663–678 (2006).

Spiers, H. J., Burgess, N., Hartley, T., Vargha-Khadem, F. & O’Keefe, J. Bilateral hippocampal pathology impairs topographical and episodic memory but not visual pattern matching. Hippocampus 11 , 715–725 (2001).

Maguire, E. A. et al. Knowing where and getting there: a human navigation network. Science 280 , 921–924 (1998).

Moser, M.-B., Rowland, D. C. & Moser, E. I. Place cells, grid cells, and memory. Cold Spring Harb. Perspect. Biol. 7 , a021808 (2015).

McAvan, A. S., Wank, A. A., Rapcsak, S. Z., Grilli, M. D. & Ekstrom, A. D. Largely intact memory for spatial locations during navigation in an individual with dense amnesia. Neuropsychologia 170 , 108225 (2022).

Finke, C., Bruehl, H., Düzel, E., Heekeren, H. R. & Ploner, C. J. Neural correlates of short-term memory reorganization in humans with hippocampal damage. J. Neurosci. Off. J. Soc. Neurosci. 33 , 11061–11069 (2013).

Byrne, P., Becker, S. & Burgess, N. Remembering the past and imagining the future: a neural model of spatial memory and imagery. Psychol. Rev. 114 , 340–375 (2007).

Bohbot, V. D., Iaria, G. & Petrides, M. Hippocampal function and spatial memory: evidence from functional neuroimaging in healthy participants and performance of patients with medial temporal lobe resections. Neuropsychology 18 , 418–425 (2004).

Feigenbaum, J. D. & Morris, R. G. Allocentric versus egocentric spatial memory after unilateral temporal lobectomy in humans. Neuropsychology 18 , 462–472 (2004).

Bohbot, V. D. et al. Spatial memory deficits in patients with lesions to the right hippocampus and to the right parahippocampal cortex. Neuropsychologia 36 , 1217–1238 (1998).

Bohbot, V. D. & Corkin, S. Posterior parahippocampal place learning in H.M. Hippocampus 17 , 863–872 (2007).

Doeller, C. F., Barry, C. & Burgess, N. Evidence for grid cells in a human memory network. Nature 463 , 657–661 (2010).

Pu, Y., Cornwell, B. R., Cheyne, D. & Johnson, B. W. The functional role of human right hippocampal/parahippocampal theta rhythm in environmental encoding during virtual spatial navigation. Hum. Brain Mapp. 38 , 1347–1361 (2016).

Suthana, N. A., Ekstrom, A. D., Moshirvaziri, S., Knowlton, B. & Bookheimer, S. Y. Human hippocampal CA1 involvement during allocentric encoding of spatial information. J. Neurosci. Off. J. Soc. Neurosci. 29 , 10512–10519 (2009).

Brookes, M. J. et al. Magnetoencephalography with optically pumped magnetometers (OPM-MEG): the next generation of functional neuroimaging. Trends Neurosci 45 , 621–634 (2022).

Delaux, A. et al. Mobile brain/body imaging of landmark-based navigation with high-density EEG. Eur. J. Neurosci. 54 , 8256–8282 (2021).

Gramann, K., Ferris, D. P., Gwin, J. & Makeig, S. Imaging natural cognition in action. Int. J. Psychophysiol. Off. J. Int. Organ. Psychophysiol. 91 , 22–29 (2014).

Kunz, L. et al. Mesoscopic neural representations in spatial navigation. Trends Cogn. Sci. 23 , 615–630 (2019).

Seymour, R. A. et al. Using OPMs to measure neural activity in standing, mobile participants. NeuroImage 244 , 118604 (2021).

Esfahani-Bayerl, N. et al. Visuo-spatial memory deficits following medial temporal lobe damage: a comparison of three patient groups. Neuropsychologia 81 , 168–179 (2016).

Braun, M. et al. Reorganization of associative memory in humans with long-standing hippocampal damage. Brain J. Neurol. 131 , 2742–2750 (2008).

Braun, M. et al. Lesions affecting the right hippocampal formation differentially impair short-term memory of spatial and nonspatial associations. Hippocampus 21 , 309–318 (2011).

Mai, J., Assheuer, J. & Paxinos, G. Atlas of the human brain . (Elsevier Academic Press, 2004).

Insausti, R. & Amaral, D. Hippocampal formation. in The human nervous system . 871–915 (Elsevier Academic Press, 2004).

Insausti, R., Tuñón, T., Sobreviela, T., Insausti, A. M. & Gonzalo, L. M. The human entorhinal cortex: a cytoarchitectonic analysis. J. Comp. Neurol. 355 , 171–198 (1995).

Insausti, R. et al. MR volumetric analysis of the human entorhinal, perirhinal, and temporopolar cortices. AJNR Am. J. Neuroradiol. 19 , 659–671 (1998).

CAS   PubMed   PubMed Central   Google Scholar  

Kallai, J., Makany, T., Karadi, K. & Jacobs, W. J. Spatial orientation strategies in Morris-type virtual water task for humans. Behav. Brain Res. 159 , 187–196 (2005).

Kothe, C. Lab streaming layer (lsl). Technical report. (2014).

Redish, A. D. Vicarious trial and error. Nat. Rev. Neurosci. 17 , 147–159 (2016).

Tao, Y. et al. A comparative analysis of trajectory similarity measures. GISci. Remote Sens . 58 , 643–669 (2021).

Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67 , 1–48 (2015).

Singmann, H. & Kellen, D. An Introduction to mixed models for experimental psychology. in New Methods in Cognitive Psychology (eds. Spieler, D. & Schumacher, E.) 4–31 (Routledge, 2019). https://doi.org/10.4324/9780429318405-2 .

Kuznetsova, A., Brockhoff, P. B. & Christensen, R. H. B. lmertest package: tests in linear mixed effects models. J. Stat. Softw. 82 , 1–26 (2017).

Lenth, R., Buerkner, P. & Herve, M. Estimated marginal means, aka least-squares means, Vol.1. 5.1. (Vienna, R Core Team, 2020).

Download references

Acknowledgements

We thank Tore Knabe for programming the water maze task. We thank Timo Berg for technical assistance. This study was funded by the Deutsche Forschungsgemeinschaft DFG, German Research Foundation—Project number 327654276—SFB 1315.

Open Access funding enabled and organized by Projekt DEAL.

Author information

These authors contributed equally: Deetje Iggena, Sein Jeung.

Authors and Affiliations

Charité - Universitätsmedizin Berlin, Department of Neurology, Augustenburger Platz 1, 13353, Berlin, Germany

Deetje Iggena, Patrizia M. Maier, Christoph J. Ploner & Carsten Finke

Humboldt-Universität zu Berlin, Berlin School of Mind and Brain, Unter den Linden 6, 10099, Berlin, Germany

Deetje Iggena, Patrizia M. Maier & Carsten Finke

Technische Universität Berlin, Department of Biological Psychology and Neuroergonomics, Fasanenstraße 1, 10623, Berlin, Germany

Sein Jeung & Klaus Gramann

Norwegian University of Science and Technology, Kavli Institute for Systems Neuroscience, Olav Kyrres gate 9,7030, Trondheim, Norway

Max-Planck Institute for Human Cognitive and Brain Sciences, Stephanstraße 1a, 04103, Leipzig, Germany

University of California, San Diego, Center for Advanced Neurological Engineering, 9500 Gilman Dr, La Jolla, CA, 92093, USA

Klaus Gramann

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: D.I., S.J., K.G., C.J.P., and C.F.; Methodology: D.I., S.J., K.G.; Participant recruitment: D.I., C.J.P., and C.F.; Data acquisition: D.I., S.J., and P.M.M.; Data analysis: D.I. and S.J.; Statistical analysis: D.I. and S.J.; Visualization: D.I.; Supervision: C.J.P., K.G., and C.F.; Writing—original draft: D.I., S.J., and C.F.; Writing—review & editing: D.I., S.J., P.M.M., K.G., C.J.P., and C.F.; Technical equipment: K.G.; Funding: C.J.P. and C.F. All authors approved the final manuscript.

Corresponding author

Correspondence to Deetje Iggena .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Communications Biology thanks the anonymous reviewers for their contribution to the peer review of this work. Primary Handling Editors: Jacqueline Gottlieb and Anam Akhtar. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Peer review file, supplementary material, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Iggena, D., Jeung, S., Maier, P.M. et al. Multisensory input modulates memory-guided spatial navigation in humans. Commun Biol 6 , 1167 (2023). https://doi.org/10.1038/s42003-023-05522-6

Download citation

Received : 05 June 2023

Accepted : 30 October 2023

Published : 14 November 2023

DOI : https://doi.org/10.1038/s42003-023-05522-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Effects of older age on visual and self-motion sensory cue integration in navigation.

• Corey S. Shayman
• Maggie K. McCracken
• Sarah H. Creem-Regehr

Experimental Brain Research (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Navigation Experiments 1

I've taken a set of SVG files and marked them up to support navigation and provided a jQuery script to provide navigation. Navigation is based on role or presence of an indicator of semantic meaning. For all the elements that are navigable, the idea is the elements should also be readable by assistive technology (AT), although no AT currently do.

Markup and navigation rules

Elements with a first child <title> or <desc> are navigable, including groups.

Elements with an aria attribute - aria-label, aria-labeledby, aria-describedby are navigable, including groups.

Groups that do not include a title, desc or aria attribute are pass through groups and are ignored for navigation. Pass through groups may exist for styling reasons, like changing the location of the text anchor or switching the fill on a set of elements.

Children of pass through groups are considered siblings of the pass through group's siblings when doing navigation.

Groups may has semantic meaning and may be included in navigation. Examples are the set of elements making up a chart axis, the set of data elements or a stack of bars in a stacked bar chart. I don't have this working well.

Roles are inherited from ancestors in this markup. A role on an element overrides the inherited role.

The following element are never navigable: title, tspan, desc, defs and any descendents of defs.

Here is a list of the roles used in navigation. Symbol used in svgTree.html is in parenthesis.

• axis (a) part of a chart axis
• background (b) *** a background, don't include in navigation or expose to AT
• connector (c) connects two items, plays the role of an SVG connector
• data (d) represents data in a chart
• event (e) an event, step, epoch of time...
• grid (gr) a reference mark, usually a line or grid marking a constant value
• group (g) a group of items with semantic meaning
• guide (gu) a guide - similar to data but represents a constant, line or curve that helps clarify data in some way
• item (i) represents a single data item in a chart
• label (L) a label
• legend (le) a chart legend
• none (x) *** presentation only, don't include in navigation or expose to AT
• note (n) a text note
• style (st) *** presentation only, don't include in navigation or expose to AT
• symbol (s) a symbol, for example a symbol in a legend entry
• title (t) a title
• view (v) a view, a top level object for navigation, for example a chart in a multiple chart SVG or a view in a technical drawing

Roles marked with *** are presentation only and should not be included in navigation or exposed to AT. Other than none, they exist to help with sighted accommodations.

• 2 HTML pages, one provides experimental tree based navigation, the other shows an SVG's structure and markup as a tree https://www.w3.org/wiki/index.php?title=File:SvgTree.zip&oldid=84178
• Marked up SVG https://www.w3.org/wiki/index.php?title=File:Markup.zip&oldid=84177
• Toggle limited content width

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: unambiguous recognition should not rely solely on natural language training.

Abstract: In LaTeX text recognition using Transformer-based architectures, this paper identifies certain "bias" issues. For instance, $e-t$ is frequently misrecognized as $e^{-t}$. This bias stems from the inherent characteristics of the dataset. To mitigate this bias, we propose a LaTeX printed text recognition model trained on a mixed dataset of pseudo-formulas and pseudo-text. The model employs a Swin Transformer as the encoder and a RoBERTa model as the decoder. Experimental results demonstrate that this approach reduces "bias", enhancing the accuracy and robustness of text recognition. For clear images, the model strictly adheres to the image content; for blurred images, it integrates both image and contextual information to produce reasonable recognition results.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

An official website of the United States government Here’s how you know keyboard_arrow_down

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Jump to main content

USPTO seeks public feedback on the current state of the experimental use exception to patent infringement

Published on: 06/27/2024 11:37 AM

At its core, the patent system aims to promote scientific, technological, and social progress; a goal advanced by both patent owners’ disclosure and publication of scientific and technical knowledge in patent documents and by the grant of robust and reliable patents that incentivize innovation and protect investment. This cycle spurs further research and development, as well as the dissemination of patented technologies that benefit society at large. The U.S. Patent and Trademark Office (USPTO) continues to strengthen our processes and procedures to help ensure that patent rights are as robust and reliable as possible, including through our work on the clarity of the record as well as our inter-agency and international work advocating for a strong patent and IP ecosystem.  A key aspect of this work is ensuring patents serve their intended purpose to attract funds, get ideas to market, and hold liable those who infringe on (or use) someone else’s patent rights.  However, well-defined exemptions to patent infringement for activities that solely constitute bona fide experimental uses, often in the context of research and development, can help preserve and amplify the function of the patent system, including for newcomers to the market and small to medium-sized entities. Courts have carved out a narrow exception for experimenting with patented subject matter, known colloquially as the “experimental use exception.” Over the years, U.S. jurisprudence has sought to define the contours of this limited exception to patent infringement. The USPTO has published a seeking public feedback on the current state of the experimental use exception jurisprudence and whether legislative action should be considered to enact a statutory experimental use exception. “Maintaining the United States as a global leader in innovation is critical for driving economic growth and job creation,” said Kathi Vidal, Under Secretary of Commerce for Intellectual Property and Director of the USPTO. “Clarifying the contours of the USPTO’s experimental use exception, especially in view of researchers’ needs today, will help us continue to address emerging challenges and unlock new opportunities in key technology sectors. This RFC will help the USPTO better understand what additional clarity, if any, researchers and other innovators need.” The agency will be accepting comments until September 26, 2024. ​ ​ ​ Visit our subscription center at to update or change your email preferences. This email was sent from an unmonitored mailbox. To contact us, please visit our website . To ensure that you continue to receive our news and notices, please modify your email filters to allow mail from . Additional information about this page Share full article Advertisement Supported by Critic’s Notebook What Happened When an Orchestra Said Goodbye to All-Male Concerts This season, the Deutsches Symphonie-Orchester Berlin experimented with programming works by female composers at every performance. Results were mixed. By Jeffrey Arlo Brown Reporting from Berlin In 2021, Marlene Brüggen, a concert planner in Germany, was listening to an episode of the podcast “Herrengedeck” and heard about a pop music festival with gender parity woven into its programming. The next day, she looked at her own festival’s planning chart, with some 200 concerts. Women were seriously underrepresented. “We hadn’t paid attention to that at all,” Brüggen said in an interview. “It was as if the bandages had been taken off my eyes.” That year, Brüggen applied for a job as director of artistic planning with the Deutsches Symphonie-Orchester Berlin. Her job interview included questions about the music she would program if hired. With her earlier epiphany in mind, she suggested the orchestra play more music by women. She got the job. Later, when she and the orchestra’s music director, Robin Ticciati, and its managing director, Thomas Schmidt-Ott, were discussing the 2023-24 season, they decided not just to include more female artists, but also to require every orchestra concert to feature at least one work written by a woman. In the fall, the orchestra plastered Berlin’s walls with posters that read “No concert without a female composer!” “The most fascinating or innovative thing about her idea wasn’t the fact of performing female composers,” Schmidt-Ott said in an interview. “It was doing it in every concert.” I went to nine performances during the season, between November and May, and heard 11 pieces by female composers. All the works were new to me, imbuing each concert with a sense of discovery unusual for an orchestra’s subscription series. We are having trouble retrieving the article content. Please enable JavaScript in your browser settings. Thank you for your patience while we verify access. If you are in Reader mode please exit and  log into  your Times account, or  subscribe  for all of The Times. Thank you for your patience while we verify access. Already a subscriber?  Log in . Want all of The Times?  Subscribe . AI & ML Hypersonics JADC2 & Comms IT/Networks Information Warfare Electronic Warfare 3D Printing The Compass Events Calendar Newsletters (Opens in new window) US Air Force opens new space lab WASHINGTON — The U.S. Air Force Research Laboratory has opened a lab focused on developing materials for new deployable space structures. Construction on the Deployable Structures Laboratory, or DeSel, began in December 2019 and opened Oct. 29, 2020. Scientists and engineers moving into the $4 million lab at Kirtland Air Force Base, New Mexico, will continue the work of the Spacecraft Component Technology Center of Excellence, which has a history of developing spacecraft structure materials. DeSel contains specialized equipment and a climate-controlled, vibration-isolated laboratory capable of testing spacecraft structures up to 20-by-15 meters, allowing AFRL to further research and develop high-strain composite materials. The Air Force’s experimental navigation satellite cleared for fabrication Navigation technology satellite–3 is an air force vanguard program being developed to demonstrate new positioning, navigation and timing technologies that will inform how future gps satellites work.. “I’m excited to have a facility that was specifically built for testing novel deployable space structures,” Benjamin Urioste, research engineer and lead for the Integrated Structural Systems team, said in a statement. “With the push toward hybrid architecture and smaller satellites, high packing efficiency structures and the ability to bring large satellite capability to small satellites is more important than ever.” “This new class of high strain composite enabled structures requires new ground test facilities,” he added. “Satellite deployments are nerve-wracking, one-shot endeavors and the high-fidelity ground testing that will take place in the DeSel is critical to ensuring on-orbit success.” Spacecraft Technologies Division chief Mark Roverse said the structures, made possible by high strain composites, "will enable new mission paradigms for the U.S. Space Force.” “AFRL has led development of high strain composites for deployed spacecraft structures, and we are excited to see the new mission capabilities that are being enabled,” Roverse added. One spacecraft in development that will benefit from these new structures is AFRL’s Space Solar Power Incremental Demonstration and Research project. With the SSPIDR project, the lab wants to build a spacecraft capable of collecting solar energy on orbit and then transmitting that energy to forward-operating bases via radio frequency. Northrop Grumman was awarded a $100 million contract to support space-based experiments informing SSPIDR’s development. The Space Force is testing that technology on the X-37B , the service’s secretive spacecraft, but DeSel’s work will provide the materials needed to build the SSPIDR craft. “This innovative research will help us to deliver the essential energy our warfighters need at the right place at the right time,” explained Col. Eric Felt, director of the AFRL’s Space Vehicles Directorate. “Some of the first structures that we look forward to testing in this new lab are those required for our Space Solar Power Incremental Demonstration and Research (SSPIDR) project, one of our top priority programs.” Nathan Strout covers space, unmanned and intelligence systems for C4ISRNET. More In Space Russia warns of possible response to US drone flights over Black Sea Russia’s defense ministry says officials have been ordered to prepare a response to u.s. drone flights over the black sea.. How achievable is the continuous Authority to Operate model? Staying current requires agile development practices that continuously integrate and deliver high-quality software with reduced risk.. Pentagon seeks objective way to test tools for zero trust compliance To increase the number of designs going through purple teaming, resnick said he wants to find a way to enlist the help of industry.. US Central Command to demo integrated counter drone sensors this fall Centcom will stage a u.s.-based exercise in october focused on spotting uas threats followed by a second demo in theater early next year.. Pentagon to issue guidance on open radio access networks to support 5G Oran is something the dod has been pushing aggressively to explore., featured video, avoiding student loan scams — money minute. U.S. troops train to tame snakes; and soldiers learn to recover fallen comrades Roundup: the biggest topics at TechNet Cyber 2024 The future of missile defense in the Pacific Trending now, army to buy more than 1,000 switchblade drones through replicator. 426 IMAGES Experimental navigation in glass phantom: A-First navigation through Royal Navy Successfully Tests Quantum Navigation System on Experimental Experimental navigation based on CIR provided by conscious pathway for Pentagon gearing up to launch 1st experimental navigation satellite in FlexMap L3Harris Technologies Clears Critical Design Review for Experimental VIDEO Experimental Navigation Mesh Sea Hunter is an autonomous unmanned surface vehicle (USV) #everyone State Estimation for Indoor and Outdoor Operation with a Micro-Aerial Vehicle China's Test Satellites Conduct Successful Experiments in Earth-Moon Communications Показываю как выглядит внутри навигационная рубка экраноплана Лунь. Эксклюзив канала в 4K! Flying MSFS COMMENTS Design Trend: Experimental Navigation Patterns While experimental navigation isn't for every design, it can be a fun alternative for the right project. 2 Million+ Digital Assets, With Unlimited Downloads. Get unlimited downloads of 2 million+ design resources, themes, templates, photos, graphics and more. Envato Elements starts at $16 per month, and is the best creative subscription we've ... 30+ Examples of Innovative and Experimental Navigation Experiences 30+ Examples of Innovative and Experimental Navigation Experiences. In this article we want to go beyond the common and safe navigation patterns that provide an easy and natural way to access content. Everybody knows that the easiest way to navigate, in terms of usability and accessibility, is to use standard patterns and UI controls like tabs ... How experimental web design can take work in new directions Traditional UI design centers around familiar, intuitive navigation. Experimental web design intentionally disrupts this routine flow. At first, site visitors might be a little unsure of what to do, like when starting a puzzle. But once visitors get used to the logic behind an experimental user interface, they can explore all that the website ... Experimental Navigation: Should You Do It? Experimental navigation offers an exciting opportunity to break away from conventional design patterns and create a truly unique user experience on your website. By embracing innovation, you can differentiate your brand, increase user engagement, and leave a lasting impression. However, it's important to balance creativity with usability and ... L3Harris delivers experimental navigation satellite By Courtney Albon. Jan 27, 2023. Not only will Navigation Technology Satellite-3 demonstrate technologies for future GPS satellites, it will actually augment the current GPS fleet while on orbit. (L3Harris image) WASHINGTON — L3Harris delivered the experimental Navigation Technology Satellite-3 to the Air Force Research Laboratory for its ... Experimental Subspace Navigation is many, many times faster than As it is, Experimental Subspace Navigation is perfectly reliable. Which seems wrong as it is Experimental. There should be a chance of failure, where the science ship and officer are lost or you arrive at a system nearby your intended target instead. Let's just say maybe a 25% chance of failure. Something that could be reduced by research down ... What is experimental subspace navigation used for? : r/Stellaris Where the route is blocked by closed borders. Simple as that. FP/FE has a few systems behind them unclaimed with no route. Experimental sub space navigation is mid-game, and cheapish to research. Jump drive is late game, rare and expensive to research. And, unlike jump drive it doesn't invite unbiden to your galaxy. Pentagon gearing up to launch 1st experimental navigation satellite in The spacecraft, called Navigation Technology Satellite-3 (NTS-3), will be the first U.S. experimental navigation satellite to take flight since NTS-1 and NTS-2 launched in the 1970s. NTS-3, which ... Experimental Navigation designs, themes, templates and ... Experimental Navigation Inspirational designs, illustrations, and graphic elements from the world's best designers. Want more inspiration? Browse our search results... View Biggest One Day Sale. Biggest One Day Sale Like. Mike Noland. Like. 6 2k View Slider Comp. Slider Comp Like. Mike Noland. Like. 2 1.8k View Dallas Landingpage ... Experimental navigation techniques for ROS robots. navigation_experimental. A collection of navigation plugins and tools: Various recovery behaviors, local and global planner plugins for move_base, a teleop filter for obstacle avoidance, and a simple control-based move_base replacement. The most useful package in this repo is sbpl_lattice_planner, a global planner plugin for move_base that ... Northrop Grumman delivers bus for Space Force's experimental navigation WASHINGTON — Northrop Grumman has delivered a bus for Navigation Technology Satellite-3 (NTS-3), ensuring that the experimental positioning satellite is on track for launch in 2023. As one of the Air Force Research Laboratory's four Vanguard programs — initiatives that are expected to deliver transformational technologies to the war ... US Space Force to test experimental navigation satellite in ... COLORADO SPRINGS, Colo. — The Air Force Research Laboratory and L3Harris will begin integrated testing this summer of an experimental satellite with implications for a future hybrid precision, navigation and timing architecture. Navigation Technology Satellite-3 (NTS-3) is being designed to showcase new PNT technology that could shape future ... Autonomous environment-adaptive microrobot swarm navigation ... Fig. 4: Experimental demonstrations of navigation autonomy levels from 0 to 2. a, The RS was manually controlled to navigate to five preset targets in sequence, which corresponds to level 0. When ... Navigation & Radios for Experimental Aircraft Navigate and communicate in the cockpit of your experimental aircraft with GPS/NAV/COMM/MFD capabilities from Garmin. ... ADS-B & Transponders Flight Decks & Displays Flight Instruments Engine Indication Systems Navigation & Radios Autopilots Audio Panels Weather Traffic Datalinks & Connectivity Portable GPS, Wearables & Apps Garmin Pilot App. navigation_experimental Wiki: navigation_experimental (last edited 2018-09-03 09:27:56 by MartinGuenther) Except where otherwise noted, the ROS wiki is licensed under the Creative Commons Attribution 3.0 GitHub A 2D navigation stack that takes in information from odometry, sensor streams, and a goal pose and outputs safe velocity commands that are sent to a mobile base. AMD64 Debian Job Status: Related stacks: An intelligent navigation experimental system based on multi-mode Visual navigation is focused more on providing the experimental steps, whereas voice navigation is focused more on the navigation of dynamically generated operations during the experimental process. 4 Experimental results and analysis 4.1 Experimental results This system mainly uses Unity3D for the design and transmits multi-mode signals to ... Multisensory input modulates memory-guided spatial navigation ... The session order of experimental setups was counterbalanced to account for potential learning effects from the first experimental setup that could influence navigation behavior in the second ... Air Force Research Lab begins integration, testing for experimental ALBUQUERQUE, N.M. — Integration and testing activities for an experimental navigation satellite are ramping up at the Air Force Research Laboratory's Space Vehicles Directorate as the U.S. Space Force prepares to launch its first major positioning, navigation and timing demonstration in nearly 50 years. The lab is on track to launch in late ... Experimental validation of the moving long base-line navigation concept This paper presents the moving long base-line (MLBL) navigation concept as well as simulation and experimental results. This multiple vehicle navigation technique consists of using vehicles fitted with accurate navigation systems as moving reference transponders to which other vehicles, fitted with less capable navigation systems, can acoustically range to update their position. Reliable ... Navigation Experiments 1 Navigation Experiments 1. I've taken a set of SVG files and marked them up to support navigation and provided a jQuery script to provide navigation. Navigation is based on role or presence of an indicator of semantic meaning. For all the elements that are navigable, the idea is the elements should also be readable by assistive technology (AT ... [2406.17148] Unambiguous Recognition Should Not Rely Solely on Natural Experimental results demonstrate that this approach reduces "bias", enhancing the accuracy and robustness of text recognition. For clear images, the model strictly adheres to the image content; for blurred images, it integrates both image and contextual information to produce reasonable recognition results. The Air Force's experimental navigation satellite cleared ... The experimental satellite will augment the GPS constellation from geosynchronous orbit, providing a geographically focused signal. "The NTS-3 Vanguard is an experimental, end-to-end demonstration of agile, resilient space-based positioning, navigation, and timing," Arlen Biersgreen, the NTS-3 program manager, said in a statement. USPTO seeks public feedback on the current state of the experimental USPTO seeks public feedback on the current state of the experimental use exception to patent infringement. At its core, the patent system aims to promote scientific, technological, and social progress; a goal advanced by both patent owners' disclosure and publication of scientific and technical knowledge in patent documents and by the grant of robust and reliable patents that incentivize ... What Happened When an Orchestra Said Goodbye to All-Male Concerts This season, the Deutsches Symphonie-Orchester Berlin experimented with programming works by female composers at every performance. Results were mixed. US Air Force opens new space lab The Air Force's experimental navigation satellite cleared for fabrication Navigation Technology Satellite-3 is an Air Force Vanguard program being developed to demonstrate new positioning, navigation and timing technologies that will inform how future GPS satellites work. essay homework paper phd project resume review speech thesis