Ask a person to find a frying pan, and they will most likely go to the kitchen. Ask a robot to do the same, and you may get numerous responses, depending on how the robot is trained.

Since humans often associate objects in a home with the room they are in, Naoki Yokoyama thinks robots that navigate human environments to perform assistive tasks should mimic that reasoning.

Roboticists have employed natural language models to help robots mimic human reasoning over the past few years. However, Yokoyama, a Ph.D. student in robotics, said these models create a “bottleneck” that prevents agents from picking up on visual cues such as room type, size, décor, and lighting. 

Yokoyama presented a new framework for semantic reasoning at the Institute of Electrical and Electronic Engineers (IEEE) International Conference on Robotics and Automation (ICRA) last month in Yokohama, Japan. ICRA is the world’s largest robotics conference.

Yokoyama earned a best paper award in the Cognitive Robotics category with his Vision-Language Frontier Maps (VLFM) proposal.

Assistant Professor Sehoon Ha and Associate Professor Dhruv Batra from the School of Interactive Computing advised Yokoyama on the paper. Yokoyama authored the paper while interning at the Boston Dynamics’ AI Institute.

“I think the cognitive robotic category represents a significant portion of submissions to ICRA nowadays,” said Yokoyama, whose family is from Japan. “I’m grateful that our work is being recognized among the best in this field.”

Instead of natural language models, Yokoyama used a renowned vision-language model called BLIP-2 and tested it on a Boston Dynamics “Spot” robot in home and office environments.

“We rely on models that have been trained on vast amounts of data collected from the web,” Yokoyama said. “That allows us to use models with common sense reasoning and world knowledge. It’s not limited to a typical robot learning environment.”

What is Blip-2?

BLIP-2 matches images to text by assigning a score that evaluates how well the user input text describes the content of an image. The model removes the need for the robot to use object detectors and language models. 

Instead, the robot uses BLIP-2 to extract semantic values from RGB images with a text prompt that includes the target object. 

BLIP-2 then teaches the robot to recognize the room type, distinguishing the living room from the bathroom and the kitchen. The robot learns to associate certain objects with specific rooms where it will likely find them.

From here, the robot creates a value map to determine the most likely locations for a target object, Yokoyama said.

Yokoyama said this is a step forward for intelligent home assistive robots, enabling users to find objects — like missing keys — in their homes without knowing an item’s location. 

“If you’re looking for a pair of scissors, the robot can automatically figure out it should head to the kitchen or the office,” he said. “Even if the scissors are in an unusual place, it uses semantic reasoning to work through each room from most probable location to least likely.”

He added that the benefit of using a VLM instead of an object detector is that the robot will include visual cues in its reasoning.

“You can look at a room in an apartment, and there are so many things an object detector wouldn’t tell you about that room that would be informative,” he said. “You don’t want to limit yourself to a textual description or a list of object classes because you’re missing many semantic visual cues.”

While other VLMs exist, Yokoyama chose BLIP-2 because the model:

• Accepts any text length and isn’t limited to a small set of objects or categories.
• Allows the robot to be pre-trained on vast amounts of data collected from the internet.
• Has proven results that enable accurate image-to-text matching.

Home, Office, and Beyond

Yokoyama also tested the Spot robot to navigate a more challenging office environment. Office spaces tend to be more homogenous and harder to distinguish from one another than rooms in a home. 

“We showed a few cases in which the robot will still work,” Yokoyama said. “We tell it to find a microwave, and it searches for the kitchen. We tell it to find a potted plant, and it moves toward an area with windows because, based on what it knows from BLIP-2, that’s the most likely place to find the plant.”

Yokoyama said as VLM models continue to improve, so will robot navigation. The increase in the number of VLM models has caused robot navigation to steer away from traditional physical simulations.

“It shows how important it is to keep an eye on the work being done in computer vision and natural language processing for getting robots to perform tasks more efficiently,” he said. “The current research direction in robot learning is moving toward more intelligent and higher-level reasoning. These foundation models are going to play a key role in that.”

Top photo by Kevin Beasley/College of Computing.

Knitting, the age-old craft of looping and stitching natural fibers into fabrics, has received renewed attention for its potential applications in advanced manufacturing. Far beyond their use for garments, knitted textiles are ideal for designing and fabricating emerging technologies like wearable electronics or soft robotics — structures that need to move and bend. 

Knitting transforms one-dimensional yarn into two-dimensional fabrics that are flexible, durable, and highly customizable in shape and elasticity. But to create smart textile design techniques that engineers can use, understanding the mechanics behind knitted materials is crucial. 

Physicists from the Georgia Institute of Technology have taken the technical know-how of knitting and added mathematical backing to it. In a study led by Elisabetta Matsumoto, associate professor in the School of Physics, and Krishma Singal, a graduate researcher in Matsumoto’s lab, the team used experiments and simulations to quantify and predict how knit fabric response can be programmed. By establishing a mathematical theory of knitted materials, the researchers hope that knitting — and textiles in general — can be incorporated into more engineering applications.

Their research paper, “Programming Mechanics in Knitted Materials, Stitch by Stitch,” was published in the journal Nature Communications. 

“For centuries, hand knitters have used different types of stitches and stitch combinations to specify the geometry and ‘stretchiness’ of garments, and much of the technical knowledge surrounding knitting has been handed down by word of mouth,” said Matsumoto.

But while knitting has often been dismissed as unskilled, poorly paid “women’s work,” the properties of knits can be more complex than traditional engineering materials like rubbers or metals. 

For this project, the team wanted to decode the underlying principles that direct the elastic behavior of knitted fabrics. These principles are governed by the nuanced interplay of stitch patterns, geometry, and yarn topology — the undercrossings or overcrossings in a knot or stitch. "A lot of yarn isn’t very stretchy, yet once knit into a fabric, the fabric exhibits emergent elastic behavior," Singal said. 

“Experienced knitters can identify which fabrics are stretchier than others and have an intuition for its best application,” she added. “But by understanding how these fabrics can be programmed and how they behave, we can expand knitting’s application into a variety of fields beyond clothing.”

Through a combination of experiments and simulations, Matsumoto and Singal explored the relationships among yarn manipulation, stitch patterns, and fabric elasticity, and how these factors work together to affect bulk fabric behavior. They began with physical yarn and fabric stretching experiments to identify main parameters, such as how bendable or fluffy the yarn is, and the length and radius of yarn in a given stitch. 

They then used the experiment results to design simulations to examine the yarn inside a stitch, similar to an X-ray. It is difficult to see inside stitches during the physical measurements, so the simulations are used to see what parts of the yarn have interacted with other parts. The simulations are used to recreate the physical measurements as accurately as possible.

Through these experiments and simulations, Singal and Matsumoto showed the profound impact that design variations can have on fabric response and uncovered the remarkable programmability of knitting. "We discovered that by using simple adjustments in how you design a fabric pattern, you can change how stretchy or stiff the bulk fabric is," Singal said. "How the yarn is manipulated, what stitches are formed, and how the stitches are patterned completely alter the response of the final fabric."

Matsumoto envisions that the insights gleaned from their research will enable knitted textile design to become more commonly used in manufacturing and product design. Their discovery that simple stitch patterning can alter a fabric’s elasticity points to knitting’s potential for cutting-edge interactive technologies like soft robotics, wearables, and haptics.

“We think of knitting as an additive manufacturing technique — like 3D printing, and you can change the material properties just by picking the right stitch pattern,” Singal said.

Matsumoto and Singal plan to push the boundaries of knitted fabric science even further, as there are still numerous questions about knitted fabrics to be answered. 

"Textiles are ubiquitous and we use them everywhere in our lives," Matsumoto said. "Right now, the hard part is that designing them for specific properties relies on having a lot of experience and technical intuition. We hope our research helps make textiles a versatile tool for engineers and scientists too."

 

Note: Sarah Gonzalez (Georgia Tech) and Michael Dimitriyev (Texas A&M) are also co-first authors of the study. 

Citation: Singal, K., Dimitriyev, M.S., Gonzalez, S.E. et al. Programming mechanics in knitted materials, stitch by stitch. Nat Commun 15, 2622 (2024). 

DOI: https://doi.org/10.1038/s41467-024-46498-z

Funding: Research Corporation for Science Advancement, National Science Foundation, and the Alfred P. Sloan Foundation 

How much does a loan officer’s familiarity with an auditor affect their client’s ability to receive a commercial loan? New research from Georgia Tech suggests that while knowing an auditor doesn’t guarantee a commercial loan, loan officers are more likely to deny loans to companies that have auditors with poor reputations. 

Arnold Schneider, a professor in the Scheller College of Business, found that loan officers who knew the borrower’s audit firm were reassured the loan wasn’t high-risk. However, familiarity with an audit firm didn’t guarantee an approved loan. If anything, an audit firm’s negative reputation for association with a defaulting client or a client needing regulatory enforcement meant a new client is more likely to lose out on a loan.

The research was presented in “Do Familiarity With a Loan Applicant’s Auditor and the Auditor’s Associations With Past Borrowers Impact Lending Judgments?” in the 2023 issue of Advances in Accounting Behavioral Research.

In his study, Schneider conducted an experiment with 64 loan officers from 49 banks across the eastern U.S. He examined two main variables: whether the loan officer was familiar or unfamiliar with an auditor and whether an auditor had a history with regulatory enforcement of a client or a defaulting client.

Each questionnaire used the same case scenario: a client applying for a $4 million commercial loan at a hypothetical loan company, along with their financial statements. Participants were asked to assess risk on a 10-point scale and provide the probability that they would extend the credit at a reasonable rate of interest. Then they rated the importance of the factors that helped them make their decisions. 

“I found that the lenders did assess the risk higher in the case where the auditors were not familiar to them,” Schneider said. “But to my surprise, that did not translate into a lower probability of granting credit to these borrowers.”

Rather, if the loan officer only knew of an auditor because of their poor history with defaulting clients or clients having regulatory enforcement, this negative association would inhibit a loan.

Understanding the role of auditors in loan decisions is crucial for borrowers. While familiarity with a loan provider can reduce perceived risk, loan approval remains a complex process. 

A new in-depth analysis shows that users who reply to misinformation about the Covid-19 vaccine on X, formerly known as Twitter, with a positive attitude, politeness, and strong evidence are more likely to encourage others to disbelieve the incorrect information.

Researchers from three Georgia Tech schools found the most effective way to confront vaccine misinformation on the X platform. 

They also created a predictive tool to show users whether their reply will succeed in changing minds or backfire and reinforce the misinformation. It can also pinpoint well-meaning replies meant to contradict misinformation but that interfere with social correction. 

A research paper with the full findings will be presented this week at the ACM Web Science Conference in Stuttgart, Germany.

Like white blood cells attacking a virus, social media users have been known to band together and debunk online misinformation being spread online in a phenomenon researchers call social correction. 

The success rate of social correction on most social media sites has not been determined. However, researchers now have a clearer picture of how successful user input can be on X. 

Their method uses a blend of artificial intelligence with a dataset of 1.5 million tweets containing misinformation about the Covid-19 vaccine. The researchers then studied user replies to misinformation as well as the consequences of those replies. 

In the paper, the researchers write that their data set pre-dates the rollout of X’s community notes feature, which allows users to submit corrections to posts on the platform. They point out that this system restricts users from responding to fact-checking text and labels and does not reflect the large flow of information on the site. 

As one of the first taxonomies of user social correction on the X platform, the researchers hope will aid future fact-checking efforts. While the paper only focused on text posts in the English language, it is a framework that can be expanded to address the growing threat of misinformation online. 

Corrective or Backfire: Characterizing and Predicting User Response to Social Correction was co-authored by Ph.D. students Bing He and Yingchen (Eric) Ma and their advisors Regents’ Entrepreneur Mustaque Ahamad, a professor with joint appointments in the School of Cybersecurity and Privacy and the School of Computer Science, and School of Computational Science and Engineering Assistant Professor Srijan Kumar. 

Melecio, who has also served as the deep tech catalyst in the Enterprise Innovation Institute’s ATDC startup incubator, will lead Innovation Lab, which encompasses new business development efforts in life sciences and biosciences. The Innovation Lab initiative centers on three core activities:

• Grow healthcare research, innovation, and workforce development practice. 
• Expand EI2 Global's international footprint. 
• Support VentureLab's National Science Foundation I-Corps activities.

“Nakia has been instrumental in helping to expand Georgia’s life sciences community and ecosystem,” said David Bridges, vice president of the Enterprise Innovation Institute, Georgia Tech’s chief economic development arm. “Leading Innovation Lab already builds on a foundation he created since joining us in 2019 and further supports our broad economic development mission.”

He's already leading in the healthcare research practice expansion with his work in the MedTech Center and running the ScaleUp Lab Program for deep tech innovation.

Under Melecio’s leadership as founding director, the MedTech Center, which has the Georgia Manufacturing Extension Partnership and Global Center for Medical Innovation as partners, has worked with and evaluated the innovations of more than 200 companies. Since launching in 2021, the MedTech Center’s 66 active startups have raised $13.1 million in investment capital and an additional $6.4 million in federal, non-dilutive funding grants.

In 2023, the MedTech Center was selected to join the Advanced Research Projects Agency for Health’s ARPA-H Investor Catalyst Hub to accelerate the commercialization of practical, accessible biomedical solutions.

He is supporting Georgia Tech’s efforts to collaborate with Atlanta University Center schools — Spelman College, Clark-Atlanta University, Morehouse College, and the Morehouse School of Medicine — to collaborate with those minority-serving institutions as they build out capacity for their scientists and researchers to create more life sciences technology companies, following an award from the Economic Development Administration.

Similarly, Melecio is working with the University of Alabama at Birmingham on a collaborative project in biologics and medical devices to move more of its researchers’ innovations out of the lab and into commercial markets.

As Innovation Lab lead, Melecio, who has secured more than $5.76 million in federal grants and awards to Georgia Tech, will also work to develop biomanufacturing partnerships for Georgia Tech.

With EI2 Global, the Enterprise Innovation Institute’s program that fosters economic opportunity through collaborations with universities, innovators, governments, and nonprofit organizations worldwide, Melecio will serve as an instructor on Lab-to-Market and CREATE-X programming for entrepreneurs. He will also create and provide educational content for EI2 Global’s university and ecosystem partners.

Closer to home, his Innovation Lab work includes ongoing projects as a principal in VentureLab, a program of Georgia Tech’s Office of Commercialization. In that capacity, he will work on VentureLab’s National Science Foundation-related Innovation Corps (I-Corps) programming. Those efforts, overseen by Commercialization Vice President Raghupathy "Siva" Sivakumar, include the NSF I-Corps Hub Academy, where Melecio will serve as director.

“Our efforts with Innovation Lab are centered around finding new opportunities, new markets, and new industries by leveraging our areas of expertise at the Enterprise Innovation Institute and Georgia Tech to build economic development capacity in the life sciences and biosciences space,” Melecio said.

“We’re looking to take a broader perspective, away from being hyper-focused in one or two niche areas in life sciences, to ensure that we maximize opportunities to support new ideas, build stronger practice areas in this space, and secure funding to bring those innovations to scale.”

Georgia Tech has been selected as one of six universities globally to receive funding for the newly established Global Industrial Technology Cooperation Center. The announcement was made by the Ministry of Trade, Industry, and Energy in South Korea during the Global Open Innovation Strategy Meeting in April.

The KIAT-Georgia Tech Semiconductor Electronics Center will receive $1.8 million to establish a sustainable semiconductor electronics research partnership between Korean companies, researchers, and Georgia Tech. 

“I am thrilled to announce that we have secured funding to launch a groundbreaking collaboration between Georgia Tech’s world-class researchers and Korean companies,” said Hong Yeo, associate professor and Woodruff Faculty Fellow in the George W. Woodruff School of Mechanical Engineering and the Wallace H. Coulter Department of Biomedical Engineering. “This initiative will drive the development of cutting-edge technologies to advance semiconductor, sensors, and electronics research.”

Yeo will lead the center, and Michael Filler, interim executive director for the Institute of Electronics and Nanotechnology, and Muhannad Bakir, director of the 3D Advanced Packaging Research Center, will serve as co-PIs.

The center will focus on advancing semiconductor research, a critical area of technology that forms the backbone of modern electronics.

The Cooperation Center is a global technology collaboration platform designed to facilitate international joint research and development planning, partner matching, and local support for domestic researchers. The selection of Georgia Tech underscores the Institute’s leadership and expertise in the field of semiconductors.