Enhancing the ability of scientists to use the latest computing and data tools will help quicken the pace of scientific discoveries

RENCI scientists and collaborators from Cornell University and University of Southern California (USC) have been awarded a $1 million, three-year grant from the National Science Foundation (NSF) to develop an innovative training program for scientists who use the Cornell High Energy Synchrotron Source (CHESS) X-ray facility. The program will be designed to help the scientists increase their computing skills, awareness and literacy with an ultimate goal of accelerating scientific innovations in synchrotron X-ray science.

A RENCI team headed by Anirban Mandal, assistant director of the Network Research & Infrastructure Group (NRIG), will lead the CyberInfrastructure Training and Education for Synchrotron X-Ray Science (X-CITE) project. It will bring together experts in cyberinfrastructure, X-ray science and other related areas from RENCI, Cornell University and USC to develop an innovative training program for researchers using CHESS, an NSF-supported high-intensity X-ray source at Cornell. CHESS is used to conduct research in materials science, physics, chemistry, biology, environmental science and other areas.

“Scientists don’t always have the computing and data expertise necessary to fully harness the instruments, data and computing tools available to transform data into insights and knowledge,” said Mandal. “We want to help reduce barriers so that scientists can effectively utilize computing capabilities and data resources at CHESS as well as cyberinfrastructure resources available through national computing and data services.”

Teaching scientists about computing tools

To get scientists up to speed on computing and data tools, the training program will cover programming essentials, systems fundamentals, distributed computing with the cyberinfrastructure ecosystem, X-ray science software and issues of data curation and applying the FAIR data principles of findability, accessibility, interoperability and reusability.

“As scientific instruments have become more sophisticated, there has been an explosion in the volume and rate of data produced by scientific facilities like CHESS,” said Mandal. “The data generated no longer fits on a laptop, and there are now computational models and AI methods that scientists can use to steer experiments based on the results they are getting. It is very difficult for scientists to keep pace with all these new capabilities.”

Mandal points out that it is important for scientists to get up to date on FAIR principles because federal research funding agencies are planning to roll out new mandates requiring scientists to share the data they generate. This will require designing metadata and figuring out how to push data into repositories in a way that makes it findable and usable by other researchers — tasks that scientists might not be accustomed to doing.

Drawing on RENCI’s expertise

The RENCI team will focus on developing common computer science modules for Python and other programming languages. This work will leverage RENCI’s expertise in this area, including Senior Research Software Developer Erik Scott’s experience as an instructor for the student program within the CI Compass project. The USC team, led by Research Professor of Computer Science Ewa Deelman, will contribute distributed computing training materials. Training materials for the specialized X-ray science software used at CHESS will be the focus of the Cornell team, which is led by Matthew Miller, associate director of CHESS.

The X-Cite training materials and activities will be available in several formats, including self-paced modules, videos, cyberinfrastructure catalogs, in-person instruction sessions, CHESS user workshops and tutorials offered at scientific conferences. The project team will also develop a coordination network to help disseminate the training materials, communicate the cyberinfrastructure needs for the X-ray science community and discuss best practices for training.

Annual short-course series aims to bridge the data literacy gap

Now in its tenth year, Data Matters, a week-long series of one and two-day courses aimed at students and professionals in business, research, and government, will take place August 7 – 11, 2023 virtually via Zoom. This short course series is sponsored by the Odum Institute for Research in Social Science at UNC-Chapel Hill, the National Consortium for Data Science, and RENCI.

In recent years, employers’ expectations for a data literate workforce have grown significantly. According to a 2022 Forrester Research Report, 70% of workers are expected to use data heavily in their jobs by 2025 – up from only 40% in 2018. Data Matters recognizes this rapidly changing data landscape and provides attendees the chance to learn from expert instructors about a wide range of topics in data science, analytics, visualization, and more.

“Upskilling is critical to maintaining a competitive workforce in today’s economy. With the rapid increase of data science tools being used in sectors such as business, research and government, it is essential that workers seek out educational opportunities that empower them to address new challenges in their field,” said Amanda Miller, associate director of the National Consortium for Data Science. “Our short-course series has fifteen courses that can be tailored to achieve individual data science goals, whether registrants are looking to refresh their knowledge or trying to learn something new in a welcoming, understanding environment.”

Data Matters instructors are experts in their fields from UNC-Chapel Hill, NC State University, Duke University, NC Central University, UT San Antonio, Oklahoma State, and more. This year’s topics include information visualization, deep learning, exploratory data analysis, statistical machine learning, artificial intelligence, and more, with classes such as:

• Basic Statistics in R, Vanessa Miller. This course focuses on analyzing a dataset to answer a research question. Students will get hands-on practice with selecting the statistical procedure to answer a research question, performing the appropriate statistical test, and interpreting the output. May be of particular interest to those who are switching to R from another program such as SAS or Stata.

• Advanced Visualization in R: R Shiny, Angela Zoss. This course will cover the basics of creating R-based web applications with Shiny, an R package that blends data science and statistical operations with interactive interface components. Participants will learn to connect interactive inputs with R operations, develop skills in web application design, and explore different options for hosting Shiny applications on the web. Basic familiarity with R is required.

• Overview of AI and Deep Learning, Siobhan Day Grady. Many key advances in AI are due to the advances in machine learning, especially deep learning. Natural language processing, computer vision, speech translation, biomedical imaging, and robotics are some areas benefiting from deep learning methods. We will look at the history of neural networks, how advances in data collection and computing caused the revival in neural networks, the different types of deep learning networks and their applications, and tools and software available to design and deploy deep networks. 

• Geospatial Analytics, Laura Tateosian. This course will focus on how to explore, analyze, and visualize geospatial data. Using Python and ArcGIS Pro, students will inspect and manipulate geospatial data, use powerful GIS tools to analyze spatial relationships, link tabular data with spatial data, and map data. In these activities, participants will use Python and the arcpy library to invoke key GIS tools for spatial analysis and mapping.

Data Matters offers reduced pricing for faculty, students, and staff from academic institutions and for professionals with nonprofit organizations. Head to the Data Matters website to register and to see detailed course descriptions, course schedules, instructor bios, and logistical information. 

Registration is now open at datamatters.org. The deadline for registration is August 3 for Monday/Tuesday courses, August 5 for Wednesday courses, and August 6 for Thursday/Friday courses.

About the National Consortium for Data Science (NCDS)

The National Consortium for Data Science (NCDS) is a collaboration of leaders in academia, industry, and government formed to address the data challenges and opportunities of the 21st century. The NCDS helps members take advantage of data in ways that result in new jobs and transformative discoveries. The organization connects diverse communities of data science experts to support a 21st century data-driven economy by building data science career pathways and creating a data-literate workforce, bridging the gap between data scientists in the public and private sectors, and supporting open and democratized data. Learn more at datascienceconsortium.org/.

The NCDS is administered by founding member RENCI, a research institute for data science and applications of cyberinfrastructure at the University of North Carolina at Chapel Hill. For more information about RENCI, visit renci.org.

The worldwide iRODS community will gather in Chapel Hill, NC from June 13 – 16

Members of the iRODS user community will meet at UNC-Chapel Hill in North Carolina for the 15th Annual iRODS User Group Meeting to participate in four days of learning, sharing use cases, and discussing new capabilities that have been added to iRODS in the last year.

The event, sponsored by RENCI, Omnibond, Globus, and Hays, will provide in-person and virtual options for attendance. An audience of over 100 participants representing dozens of academic, government, and commercial institutions is expected to join.

“The robust list of presentations at the 2023 iRODS User Group Meeting illustrates the impact and utility of iRODS on a global scale, from talks on unique applications of the software to demos of innovative clients and integrations.” said Terrell Russell, executive director of the iRODS Consortium. “We are excited to host the user community in our hometown of Chapel Hill, NC and provide opportunities for learning, networking, and collaboration throughout the week.”

In May, the iRODS Consortium published the 2023 Technology Roadmap, which documents the state of the technical direction chosen for the iRODS data management software. A notable focus named in this plan is to make implementing the iRODS Protocol less complicated by designing a new HTTP API.

Other plans include updates to the GenQuery interface and the iCommands client. The iRODS GenQuery interface has long defined the way users and administrators can search the iRODS namespace, its storage systems, users, and metadata, while honoring the iRODS permission model. The next generation of GenQuery, GenQuery2, is now available for experimentation. The current iRODS iCommands are a culmination of many years of effort, but they are beginning to show their age, especially in terms of design and extensibility. The iRODS Consortium aims to create a brand new CLI that focuses on using modern libraries (iRODS or otherwise), modern C++, being extensible and modular, and providing a single binary.

During last year’s UGM, users learned about a new PAM (pluggable authentication module) plugin, a universal implementation for all authentication methods. At this year’s event, the iRODS Consortium will provide an overview and demonstration of exploratory work with further authentication services such as OAuth 2.0, OpenID Connect, and the new iRODS HTTP API, and how integrations with these services may work best in the future.

As always with the annual UGM, in addition to general software updates, users will offer presentations about their organizations’ deployments of iRODS. This year’s meeting will feature over 15 talks from users around the world. Among the use cases and deployments to be featured are:

• GoCommands: A cross-platform command-line client for iRODS. CyVerse / University of Arizona. The diversity of scientific computing platforms has increased significantly, ranging from small devices like Raspberry Pi to large computing clusters. However, accessing iRODS data on these varied platforms remains a common but challenging requirement. The official command-line tool for iRODS, iCommands, is limited to a few platforms like CentOS7 and Ubuntu 18/20. As a result, users on other platforms like MacOS, Windows, and Raspberry Pi OS have no straightforward performant means of accessing iRODS. GoCommands is another command-line tool for iRODS designed to address the portability issue of iCommands. Written in Go programming language, building its executable for diverse platforms is straightforward. The tool is a single executable that does not require any dependency installation. Pre-built binaries for MacOS, Linux (any distros), and Windows, regardless of their CPU architectures, are already available. In addition, the tool does not require elevated privileges for installation and run. This makes it possible for users on nearly any platform to access iRODS.
• iBridges: A comprehensive way of interfacing with iRODS. Utrecht University, Wageningen University, and University of Groningen. iRODS is a rich middleware providing means to facilitate data management for research. It implements all necessary concepts like resources, metadata, permissions, and rules. However, in research most of the concepts are still new. Hence, researchers and their support staff are challenged using the current interfaces and tools to 1) learn about those concepts and 2) familiarize themselves with the different APIs and command line interfaces. This creates the need for a steep learning curve for researchers and research supporters, slowing down the adoption of iRODS. To ease the usage of iRODS we present iBridges. iBridges is a standalone desktop application, written in Python, to provide users of Windows, Linux, and MacOS with a graphical user interface (GUI) to interact with iRODS servers. The tool is agnostic to any rules/policies in the server. Out-of-the-box iBridges supports three main functions: browsing and manipulating data objects, upload/download data, and searching data collections.
• ManGO: A web portal and framework built on top of iRODS for active research data management. KU Leuven. At the University of Leuven. Belgium, we are building the infrastructure and software layers to leverage iRODS as a major building block in active research data management. This involves various workflows and processing of data and metadata during the lifetime of a research project. One of the important components consists of a modular and adaptable web portal built using the iRODS Python client. Given the wide range of use cases, the web framework employs some classical architectural patterns to decouple specialized domain specific needs from the core system. It also has features that make it behave like a content management system, including a (view) template override system that make the representation of collections and data objects dependent on for example specific metadata or collection structure. Metadata is a prime focus to steer many aspects of this framework along its core use for research data, and a considerable effort was also put in a user friendly metadata schema management system. In this talk, we will present the current status as well as near future plans.
• iRODS Object Store on Galaxy Server: Application of iRODS to a Real Time, Multi-user System. Penn State University. Galaxy is an open-source platform for data analysis that enables users to 1) Use tools from various domains through its graphical web interface, 2) Run code in interactive environments such as Jupyter or RStudio, 3) Manage data by sharing and publishing results, workflows, and visualizations, and 4) Ensure reproducibility by capturing the necessary information to repeat data analyses. To store data Galaxy utilizes ObjectStore as its data virtualization layer. It abstracts Galaxy’s domain logic for data persistence technology. Currently, Galaxy mainly uses a Disk ObjectStore for data persistence. To extend Galaxy’s data persistence capabilities, we had previously extended Galaxy’s ObjectStore to support iRODS. In this work, we discuss the steps in deploying iRODS Object Store on the USA-based Galaxy server (usegalaxy.org) and the challenges we faced. To the best of our knowledge, after CyVerse, this is one of the few applications of iRODS to a real time, multi-user system.

Bookending this year’s UGM are two in-person events for those who hope to learn more about iRODS. On June 13, the Consortium is offering beginner and advanced training sessions. After the conference, on June 16, users have the chance to register for a troubleshooting session, devoted to providing one-on-one help with an existing or planned iRODS installation or integration.

Registration for both physical and virtual attendance will remain open until the beginning of the event. Learn more at this year’s UGM at irods.org/ugm2023. 

About the iRODS Consortium

The iRODS Consortium is a membership organization that supports the development of the integrated Rule-Oriented Data System (iRODS), free open source software for data virtualization, data discovery, workflow automation, and secure collaboration. The iRODS Consortium provides a production-ready iRODS distribution and iRODS training, professional integration services, and support. The world’s top researchers in life sciences, geosciences, and information management use iRODS to control their data. Learn more at irods.org.

The iRODS Consortium is administered by founding member RENCI, a research institute for applications of cyberinfrastructure at the University of North Carolina at Chapel Hill. For more information about RENCI, visit renci.org.

Ashok Krishnamurthy has been appointed director of the Renaissance Computing Institute (RENCI). Krishnamurthy previously served as RENCI’s deputy director and has been engaged in the leadership team that has guided RENCI over the past 10 years. In addition, he has served as interim director of RENCI since last July when previous director Stan Ahalt assumed deanship of the new UNC School of Data Science and Society.

“The momentum behind RENCI’s projects and research has grown rapidly over the last few years, and the road ahead is limitless,” said Ashok Krishnamurthy, director of RENCI. “I’m excited to step into this role and collaborate with our expanding workforce to build a collective vision and roadmap for the future.”

Krishnamurthy has decades of experience as both a researcher and an administrator, with special emphasis on forming and advancing interdisciplinary teams. He collaborates with researchers in informatics, health, and social sciences to develop projects and programs that leverage the power of data science and scalable computing to solve challenging problems. He advises students and mentors post-doctoral scholars and junior investigators and is deeply involved in managing and enhancing research partnerships with other institutions and businesses. 

Krishnamurthy holds a bachelor’s degree in electrical engineering from the Indian Institute of Technology, and a master’s and PhD in electrical and computer engineering from the University of Florida. Krishnamurthy’s research over the years has been funded by the National Science Foundation, the National Institutes of Health, the Department of Defense, the Defense Advanced Research Projects Agency, and the Department of Energy. 

The mission of RENCI is to develop and deploy data science cyberinfrastructure that helps researchers in academia, government, and business use data to drive discoveries, make informed decisions, and spur economic development. As director, Krishnamurthy is responsible for all operations that bring that mission to life, including managing five research groups of over 80 researchers and the Advanced Cyberinfrastructure Support Group (ACIS). In addition, he holds appointments as a research professor in the Department of Computer Science and as co-director for Informatics and Data Science (IDSci) at the North Carolina Translational and Clinical Sciences Institute (NC TraCS). 

“Over the next few years, in addition to our continued excellence in fields such as climate science, clinical informatics, and network research infrastructure, I see RENCI emerging as a leader in team science,” said Krishnamurthy. “With our years of experience in creating efficient interdisciplinary teams, RENCI is uniquely positioned to provide expertise on incorporating the most effective research practices, coordination and outreach efforts, and technology and tools to projects on UNC’s campus, in the Triangle, and beyond.”

“[Ashok] brings cross-cutting and highly technical expertise, deep collaborative ties across Carolina, the region, and the world, and a wonderful leadership style,” said Penny Gordon-Larsen, interim vice chancellor for research, in an email sent to UNC deans, department heads, and directors. “We are confident that under his leadership, RENCI will continue its trajectory of excellence in the development and deployment of advanced technologies to advance research discoveries and practical innovations.” 

Spectra Logic, a global leader in data management and data storage solutions, today announced a collaboration with the iRODS Consortium to create a joint solution built upon Spectra Vail^® software, Spectra BlackPearl^® S3 storage and the iRODS data management platform. The combined solution enables customers to use industry-standard cloud interfaces for on-premises disk and on-premises glacier* storage with object tape, while unlocking multi-site/multi-cloud capabilities.

The iRODS integration with BlackPearl S3 allows organizations to leverage the performance and cost benefits of on-premises glacier storage as disk or tape to access “cold” data and automate workflows, while the integration with Vail provides access to cloud services across multiple clouds. Spectra Vail software and BlackPearl S3 storage have been tested with the iRODS S3 storage resource plugin to fully support the Amazon^® S3 abstraction that iRODS delivers. The new functionality is available as part of the iRODS 4.2.11 release.

“Organizations that need an on-prem glacier tier will see many benefits with the interoperability between BlackPearl S3 and the iRODS data management platform,” said David Feller, Spectra Logic vice president of product management and solutions engineering. “Organizations will be able to take full advantage of on-prem storage and the public, private and hybrid cloud by leveraging the Vail and iRODS integration.”

“The combined Spectra Logic and iRODS solution will enable organizations that rely heavily on tape to archive petabytes of valuable digital data economically and efficiently in a glacier-like tier,” said Terrell Russell, executive director of the iRODS Consortium. “We look forward to a lasting collaboration with Spectra Logic that will help our mutual customers drive innovation and accelerate business results.”

*Amazon Glacier is a registered trademark of Amazon Technologies, Inc.

# # #

About the iRODS Consortium

The iRODS Consortium is a membership-based organization that guides development and support of iRODS as free open-source software for data discovery, workflow automation, secure collaboration, and data virtualization. The iRODS Consortium provides a production-ready iRODS distribution and iRODS professional integration services, training, and support. The consortium is administered by founding member RENCI, a research institute for applications of cyberinfrastructure located at the University of North Carolina at Chapel Hill, USA.

About Spectra Logic
Spectra Logic develops a full range of Attack Hardened™ data management and data storage solutions for a multi-cloud world. Dedicated solely to data storage innovation for more than 40 years, Spectra Logic helps organizations modernize their IT infrastructures and protect and preserve their data with a broad portfolio of solutions that enable them to manage, migrate, store and preserve business data long-term, along with features to make them ransomware resilient, whether on-premises, in a single cloud, across multiple clouds, or in all locations at once. To learn more, visit www.spectralogic.com.