Publications

Abstract

The paper analyzes and characterizes the algebraic and logical structure of the multiset semantics for SPARQL patterns involving AND, UNION, FILTER, EXCEPT, and SELECT. To do this, we align SPARQL with two well-established query languages: Datalog and Relational Algebra. Specifically, we study (i) a version of non-recursive Datalog with safe negation extended to support multisets, and (ii) a multiset relational algebra comprising projection, selection, natural join, arithmetic union, and except. We prove that these three formalisms are expressively equivalent under multiset semantics.

Abstract

Semantic Web technologies are increasingly used in the architecture, engineering, and construction (AEC) industry, yet the Resource Description Framework (RDF) and its query language, SPARQL, still lack native support for 3D geometry. Existing approaches either reduce geometry to 2D, rely on external spatial databases, or require processing workflows outside the semantic layer. This paper introduces geof3D, an extension to SPARQL that enables 3D geometric computation directly inside RDF triple stores. The framework is grounded in a formal function space derived from Architectural Geometry and provides typed operators for measurement, spatial predicates, constructive solid modeling, and affine transformations. These functions are implemented as SPARQL built-ins in RDF4J, supported by an execution backend that uses Java-based processing together with SFCGAL, a robust computational geometry engine accessed through the Java Native Interface (JNI). The system supports operations including geometric validation, Boolean solids, 3D spatial queries, and shape transformations without leaving the RDF environment. We evaluate geof3D using real building models from the Large-Scale Construction Robotics Laboratory and show that the framework supports spatial alignment, clash detection, and algorithmic modeling entirely through RDF-native queries. The evaluation examines both expressiveness and implementation performance, combining in-browser benchmarking with direct JNI measurements and comparative testing against a PostGIS configuration to assess performance, scalability, and geometric fidelity. All code, queries, datasets, and benchmarks are openly released. This work shows that SPARQL can serve not only as a semantic query language but also as a computational interface for 3D co-design, enabling integrated, interoperable, and geometry-aware workflows for building information management.

Abstract

The vast majority of Web pages fail to comply with established Web accessibility guidelines, excluding a range of users with diverse abilities from interacting with their content. Making Web pages accessible to all users requires dedicated expertise and additional manual efforts from Web page providers. To lower their efforts and promote inclusiveness, we aim to automatically detect and correct Web accessibility violations in HTML code. While previous work has made progress in detecting certain types of accessibility violations, the problem of automatically detecting and correcting accessibility violations remains an open challenge that we address. We introduce a novel taxonomy classifying Web accessibility violations into three key categories - Syntactic, Semantic, and Layout. This taxonomy provides a structured foundation for developing our detection and correction method and redefining evaluation metrics. We propose a novel method, AccessGuru, which combines existing accessibility testing tools and Large Language Models (LLMs) to detect violations and applies taxonomy-driven prompting strategies to correct all three categories. To evaluate these capabilities, we develop a benchmark of real-world Web accessibility violations. Our benchmark quantifies syntactic and layout compliance and judges semantic accuracy through comparative analysis with human expert corrections. Evaluation against our benchmark shows that AccessGuru achieves up to 84% average violation score decrease, significantly outperforming prior methods that achieve at most 50%.

Abstract

Modeling evolving interactions among entities is critical in many real-world tasks. For example, predicting driver maneuvers in traffic requires tracking how neighboring vehicles accelerate, brake, and change lanes relative to one another over consecutive frames. Likewise, detecting financial fraud hinges on following the flow of funds through successive transactions as they propagate through the network. Unlike classic time-series forecasting, these settings demand reasoning over who interacts with whom and when, calling for a temporal-graph representation that makes both the relations and their evolution explicit. Existing temporal-graph methods typically use snapshot graphs to encode temporal evolution. We introduce a full-history graph that instantiates one node for every entity at every time step and separates two edge sets: (i) intra-time-step edges that capture relations within a single frame and (ii) inter-time-step edges that connect an entity to itself at consecutive steps. To learn on this graph we design an Edge-Type Decoupled Network (ETDNet) with parallel modules: a graph-attention module aggregates information along intra-time-step edges, a multi-head temporal-attention module attends over an entity's inter-time-step history, and a fusion module combines the two messages after every layer. Evaluated on driver-intention prediction (Waymo) and Bitcoin fraud detection (Elliptic++), ETDNet consistently surpasses strong baselines, lifting Waymo joint accuracy to 75.6\% (vs. 74.1\%) and raising Elliptic++ illicit-class F1 to 88.1\% (vs. 60.4\%). These gains demonstrate the benefit of representing structural and temporal relations as distinct edges in a single graph.

Abstract

Precise optical inspection in industrial applications is crucial for minimizing scrap rates and reducing the associated costs. Besides merely detecting if a product is anomalous or not, it is crucial to know the distinct type of defect, such as a bent, cut, or scratch. The ability to recognize the "exact" defect type enables automated treatments of the anomalies in modern production lines. Current methods are limited to solely detecting whether a product is defective or not without providing any insights on the defect type, nevertheless detecting and identifying multiple defects. We propose MultiADS, a zero-shot learning approach, able to perform Multi-type Anomaly Detection and Segmentation. The architecture of MultiADS comprises CLIP and extra linear layers to align the visual- and textual representation in a joint feature space. To the best of our knowledge, our proposal, is the first approach to perform a multi-type anomaly segmentation task in zero-shot learning. Contrary to the other baselines, our approach i) generates specific anomaly masks for each distinct defect type, ii) learns to distinguish defect types, and iii) simultaneously identifies multiple defect types present in an anomalous product. Additionally, our approach outperforms zero/few-shot learning SoTA methods on image-level and pixel-level anomaly detection and segmentation tasks on five commonly used datasets: MVTec-AD, Visa, MPDD, MAD and Real-IAD.

Abstract

Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current benchmarks for CQA might not be as complex as we think, as the way they are built distorts our perception of progress in this field. For example, we find that in these benchmarks, most queries (up to 98% for some query types) can be reduced to simpler problems, e.g., link prediction, where only one link needs to be predicted. The performance of state-of-the-art CQA models decreases significantly when such models are evaluated on queries that cannot be reduced to easier types. Thus, we propose a set of more challenging benchmarks composed of queries that require models to reason over multiple hops and better reflect the construction of real-world KGs. In a systematic empirical investigation, the new benchmarks show that current methods leave much to be desired from current CQA methods.

Abstract

Uncertainty quantification in Knowledge Graph Embedding (KGE) methods is crucial for ensuring the reliability of downstream applications. A recent work applies conformal prediction to KGE methods, providing uncertainty estimates by generating a set of answers that is guaranteed to include the true answer with a predefined confidence level. However, existing methods provide probabilistic guarantees averaged over a reference set of queries and answers (marginal coverage guarantee). In high-stakes applications such as medical diagnosis, a stronger guarantee is often required: the predicted sets must provide consistent coverage per query (conditional coverage guarantee). We propose CondKGCP, a novel method that approximates predicate-conditional coverage guarantees while maintaining compact prediction sets. CondKGCP merges predicates with similar vector representations and augments calibration with rank information. We prove the theoretical guarantees and demonstrate empirical effectiveness of CondKGCP by comprehensive evaluations.

Abstract

Knowledge graph embeddings (KGE) apply machine learning methods on knowledge graphs (KGs) to provide non-classical reasoning capabilities based on similarities and analogies. The learned KG embeddings are typically used to answer queries by ranking all potential answers, but rankings often lack a meaningful probabilistic interpretation - lower-ranked answers do not necessarily have a lower probability of being true. This limitation makes it difficult to quantify uncertainty of model’s predictions, posing challenges for the application of KGE methods in high-stakes domains like medicine. We address this issue by applying the theory of conformal prediction that allows generating answer sets, which contain the correct answer with probabilistic guarantees. We explain how conformal prediction can be used to generate such answer sets for link prediction tasks. Our empirical evaluation on four benchmark datasets using six representative KGE methods validates that the generated answer sets satisfy the probabilistic guarantees given by the theory of conformal prediction. We also demonstrate that the generated answer sets often have a sensible size and that the size adapts well with respect to the difficulty of the query.

Abstract

RDF graphs and (Labeled) Property Graphs (PGs) have emerged as data models for representing graph databases. Given the differences between the two models, ensuring interoperability between them has become essential, to leverage the strengths of both models. Various approaches have been proposed to map PGs to RDF graphs. However, these approaches differ in terms of structure, representation, size of the generated RDF graph, and degree of configuration provided to the user, making direct comparisons challenging. While declarative methods prevailed to construct RDF graphs from other data formats, the mapping languages proposed for such transformations have not been considered so far for mapping PGs to RDF graphs. In this work, we provide a representation of PG-to-RDF approaches through templates described using RML, a mapping language to construct RDF graphs from heterogeneous data. We show that all considered PG-to-RDF approaches can be represented in RML and, by having a uniform representation of them, we can compare them showcasing their differences. Finally, we show that not only can RML be used to capture PG-to-RDF mappings, but it actually offers more expressive power than the considered PG-to-RDF approaches.

Abstract

The Architecture, Engineering, and Construction (AEC) industry faces data integration challenges due to fragmented silos and diverse data representations, hindering cross-domain queries and early detection of design constraints. Semantic Web Technologies (SWTs) address data integration challenges. This paper evaluates the impact of SWTs on co-design workflows by comparing them with alternative approaches to assess their effectiveness in supporting interdisciplinary collaboration and design constraint detection. Using Design Science Research, a co-design methodology is developed that integrates SWTs with AEC tools for reasoning and federated querying. A component of this methodology is a bidirectional mapping strategy for translating object-oriented data models, demonstrated with the Building Habitat Object Model (BHoM), an AEC interoperability framework. Findings reveal that integrating SWTs enables reasoning and complex queries across federated datasets, improving co-design efficiency. These findings support AEC professionals in advancing co-design and data-driven decision-making, while also informing future research on integrating SWTs into AEC design workflows.

Abstract

In the field of knowledge representation, Knowledge Graphs~(KGs) play a crucial role in organizing and representing vast amounts of information. However, they are inherently incomplete, as links can be missing from the graph. The task of predicting such missing links is known as link prediction. Link prediction is performed in different settings, namely transductive, where all entities and relation types are observed during training; inductive, where some entities are unseen; and fully-inductive, where some entities and relation types are unseen. A considerable amount of effort has been devoted to improving link prediction in the transductive setting, with Knowledge Graph Embedding~(KGE) emerging as a prominent approach. While research on inductive and fully-inductive link prediction has been conducted, existing methods rely on additional information beyond the set of training edges. To leverage the rich transductive literature for the two inductive settings, recent work in the form of BytE proposed using a BPE tokenizer to adapt transductive KGE to the inductive settings. However, it relies on a pre-constructed token vocabulary that may not be suitable for all knowledge graphs, and its evaluation is limited to benchmarks that contain only a few test edges with unseen elements. This thesis builds upon BytE by introducing purpose-built token vocabularies, modifying the process of transforming token embeddings into entity and relation embeddings, and subsequently performing a systematic evaluation on standard inductive benchmarks.

Abstract

Predicting answers to queries over knowledge graphs is called a complex reasoning task because answering a query requires subdividing it into subqueries. Existing query embedding methods use this decomposition to compute the embedding of a query as the combination of the embedding of the subqueries. This requirement limits the answerable queries to queries having a single free variable and being decomposable, which are called tree-form queries and correspond to the SROI^- description logic. In this paper, we define a more general set of queries, called DAG queries and formulated in the ALCOIR description logic, propose a query embedding method for them, called DAGE, and a new benchmark to evaluate query embeddings on them. Given the computational graph of a DAG query, DAGE combines the possibly multiple paths between two nodes into a single path with a trainable operator that represents the intersection of relations and learns DAG-DL from tautologies. We show that it is possible to implement DAGE on top of existing query embedding methods, and we empirically measure the improvement of our method over the results of vanilla methods evaluated in tree-form queries that approximate the DAG queries of our proposed benchmark.

Abstract

The shift towards online meetings, seminars, and lectures has rapidly gained momentum in recent years, particularly accelerated by the events surrounding the COVID-19 pandemic. However, the absence of nonverbal cues such as eye contact and gestures in virtual settings presents significant challenges for effective communication. Existing frameworks for virtual meetings often fail to adequately address this issue, making it difficult for presenters to accurately assess audience engagement and comprehension. This thesis investigates these challenges by capturing participant attention and emotion in real-time and evaluating how presenters interact with live feedback. Through the implementation and study of a feedback system that visualizes gaze and emotional data, we aimed to bridge the gap between in-person and online lecture experiences. To ensure an informed approach, we conducted a requirement analysis prior to our experiment, identifying key factors for effective feedback integration. In the experiment, we collected data through our tool by logging meeting interactions and analyzing responses from a post-experiment questionnaire. Our findings provide insights into the impact of such feedback on presenter motivation, delivery adjustments, and engagement levels, ultimately contributing to the improvement of educational quality in virtual environments.

Abstract

Named entity recognition (NER) is a fundamental task in Natural Language Processing (NLP) that involves identifying and classifying entities in text. NER forms the basis for solving many NLP tasks that play an important role in the biomedical domain. However, the scarcity of annotated data poses significant challenges in training effective NER models, particularly in low-resource languages, such as German. This study aims to evaluate five models in a low-resource environment, such as oncology in German, to gain insights into possible solutions to overcome this data scarcity. These approaches include prompt engineering of a Large Language Model, a BERT-based model, fine-tuned separately on the WikiMed-DE-BEL and a synthetic dataset, and a GLiNER-based model architecture. A model fine-tuned on a German gold standard oncology dataset serves as a benchmark to compare the other four models to. The evaluation is conducted using two datasets: MAED-KS, a de-identified oncology dataset consisting of patient reports collected from a German hospital, and the aluminium standard dataset, a synthetic dataset constructed based on disease co-occurrence, extracted from hospital data. Our results show that the GLiNER architecture achieves the highest F1-scores of the four approaches on the MAED-KS dataset and similar scores to the benchmark model. The Prompt Engineered Large Language Model performed best on the aluminum standard dataset. Overall, the GLiNER model shows the best performance and good adaptability in a low resource environment, such as oncology data in German.

Abstract

Symbolic regression is an interpretable machine learning method that learns mathematical expressions from given data. It naturally combines with Bayesian Inference which lets experts express their knowledge as prior distributions over equations. However, the infinite search space of mathematical expressions renders exhaustive search impractical, and Bayesian Inference remains costly. Therefore, we propose to execute the Bayesian Reasoning in the learned latent space of a trained Variational Autoencoder (VAE) and thereby exploit inherent structures in the search space. While latent spaces have been used to structure search spaces, our approach provides the probability of each mathematical expression rather than selecting the best one. We suggest practical approximations to the posterior distribution in latent space and obtain formula examples by sampling from the posterior using the Gaussian Process Hamiltonian Monte Carlo (GP-HMC) method. We have validated our method using various Koza, Nguyen, and self-generated datasets and compared it against genetic programming and SInDy concerning the Root Mean Square Error (RMSE). Keywords: Symbolic Regression, latent space, Variational Autoencoder, Character Variational Autoencoder, Grammar Variational Autoencoder, Bayesian Reasoning, Gaussian Process, Hamiltonian Monte Carlo, Gaussian Process Hamiltonian Monte Carlo.

Abstract

New economic systems are needed to decouple resource consumption from wealth. The linear economic approach of “take-make-use-dispose” is not a recipe for success in the long term. Circular production offers a solution to this problem. Our Collaborative Research Center 1574 aims to enable integrated linear and circular production on an industrial scale. To this end, the Collaborative Research Center 1574 investigates how used products and their multiple generations can achieve the vision of the perpetual product. This research involves multiple scientific questions related to production technology, product development and materials technology, ergonomics, robotics, computer science, and knowledge modeling. The Collaborative Research Center 1574 involves eighteen subprojects. Each one studies a dimension of these multiple scientific questions. One of these subprojects, called INF, aims to provide an infrastructure and teach the other subprojects’ researchers how to operate and integrate all the data they produced into a unified knowledge graph. This paper describes the roadmap of the INF subproject, including the ongoing work, future steps, and vision of the INF subproject.

Abstract

Retrieval-Augmented Generation (RAG) enhances large language models by incorporating external knowledge, yet suffers from critical limitations in high-stakes domains -- namely, sensitivity to noisy or contradictory evidence and opaque, stochastic decision-making. We propose ArgRAG, an explainable, and contestable alternative that replaces black-box reasoning with structured inference using a Quantitative Bipolar Argumentation Framework (QBAF). ArgRAG constructs a QBAF from retrieved documents and performs deterministic reasoning under gradual semantics. This allows faithfully explaining and contesting decisions. Evaluated on two fact verification benchmarks, PubHealth and RAGuard, ArgRAG achieves strong accuracy while significantly improving transparency.

Abstract

Over the past few years, we have seen the emergence of large knowledge graphs combining information from multiple sources. Sometimes, this information is provided in the form of assertions about other assertions, defining contexts where assertions are valid. A recent extension to RDF which admits statements over statements, called RDF-star, is in revision to become a W3C standard. However, there is no proposal for a semantics of these RDF-star statements nor a built-in facility to operate over them. In this paper, we propose a query language for epistemic RDF-star metadata based on a four-valued logic, called eSPARQL. Our proposed query language extends SPARQL-star, the query language for RDF-star, with a new type of FROM clause to facilitate operating with multiple and sometimes conflicting beliefs. We show that the proposed query language can express four use case queries, including the following features: (i) querying the belief of an individual, (ii) the aggregating of beliefs, (iii) querying who is conflicting with somebody, and (iv) beliefs about beliefs (i.e., nesting of beliefs).

Abstract

Query embedding approaches answer complex logical queries over incomplete knowledge graphs (KGs) by computing and operating on low-dimensional vector representations of entities, relations, and queries. However, current query embedding models heavily rely on excessively parameterized neural networks and cannot explain the knowledge learned from the graph. We propose a novel query embedding method, AConE, which explains the knowledge learned from the graph in the form of SROI− description logic axioms while being more parameter-efficient than most existing approaches. AConE associates queries to SROI− description logic concepts. Every SROI− concept is embedded as a cone in complex vector space, and each SROI− relation is embedded as a transformation that rotates and scales cones. We show theoretically that AConE can learn SROI− axioms, and defines an algebra whose operations correspond one-to-one to SROI− description logic concept constructs. Our empirical study on multiple query datasets shows that AConE achieves superior results over previous baselines with fewer parameters. Notably on the WN18RR dataset, AConE achieves significant improvement over baseline models. We provide comprehensive analyses showing that the capability to represent axioms positively impacts the results of query answering.

Abstract

Despite significant advances in Semantic Role Labeling (SRL), much work in this field has been carried out with a focus on verbal predicates, with the research on nominal SRL lagging behind. In many contexts, however, nominal predicates are often as informative as verbal ones, thus needing proper treatment. In this paper we aim to fill this gap and make nominal SRL a first-class citizen. We introduce a novel approach to create the first large-scale, high-quality inventory of nominal predicates and organize them into semantically-coherent frames. Although automatically created, NounAtlas -- our frame inventory -- is subsequently fully validated. We then put forward a technique to generate silver training data for nominal SRL and show that a state-of-the-art SRL model can achieve good performance. Interestingly, thanks to our design choices which enable seamless integration of our predicate inventory with its verbal counterpart, we can mix verbal and nominal data and perform robust SRL on both types of predicates.

Abstract

SPARQL CONSTRUCT queries allow for the specification of data processing pipelines that transform given input graphs into new output graphs. Input graphs are now commonly constrained through SHACL shapes allowing for both their validation and aiding users (as well as tools) in understanding their structure. However, it becomes challenging to understand what graph data can be expected at the end of a data processing pipeline without knowing the particular input data: Shape constraints on the input graph may affect the output graph, but may no longer apply literally, and new shapes may be imposed by the query itself. In our recent work, From Shapes to Shapes: Inferring SHACL Shapes for Results of SPARQL CONSTRUCT Queries, we studied the derivation of shape constraints that hold on all possible output graphs of a given SPARQL CONSTRUCT query by axiomatizing the query and the shapes with the ALCHOI description logic. This extended abstract summarizes our previous work.

Abstract

The popularity of Knowledge Graphs (KGs) both in industry and academia owes credit to their flexible data model, suitable for data integration from multiple sources. Several KG-based applications such as trust assessment or view maintenance on dynamic data rely on the ability to compute provenance explanations for query results. The how-provenance of a query result is an expression that encodes the records (triples or facts) that explain its inclusion in the result set. This article proposes NPCS, a Native Provenance Computation approach for SPARQL queries. NPCS annotates query results with their how-provenance. By building upon spm-provenance semirings, NPCS supports both monotonic and non-monotonic SPARQL queries. Thanks to its reliance on query rewriting techniques, the approach is directly applicable to already deployed SPARQL engines using different reification schemes -- including RDF-star. Our experimental evaluation on two popular SPARQL engines (GraphDB and Stardog) shows that our novel query rewriting brings a significant runtime improvement over existing query rewriting solutions, scaling to RDF graphs with billions of triples.

Abstract

With the widespread adoption of machine learning systems, the need to curtail their behavior has become increasingly apparent. This is evidenced by recent advancements towards developing models that satisfy robustness, safety, and fairness requirements. These requirements can be imposed (with generalization guarantees) by formulating constrained learning problems that can then be tackled by dual ascent algorithms. Yet, though these algorithms converge in objective value, even in non-convex settings, they cannot guarantee that their outcome is feasible. Doing so requires randomizing over all iterates, which is impractical in virtually any modern applications. Still, final iterates have been observed to perform well in practice. In this work, we address this gap between theory and practice by characterizing the constraint violation of Lagrangian minimizers associated with optimal dual variables, despite lack of convexity. To do this, we leverage the fact that non-convex, finite-dimensional constrained learning problems can be seen as parametrizations of convex, functional problems. Our results show that rich parametrizations effectively mitigate the issue of feasibility in dual methods, shedding light on prior empirical successes of dual learning. We illustrate our findings in fair learning tasks.

Abstract

We introduce Graph-Induced Sum-Product Networks (GSPNs), a new probabilistic framework for graph representation learning that can tractably answer probabilistic queries. Inspired by the computational trees induced by vertices in the context of message-passing neural networks, we build hierarchies of sum-product networks (SPNs) where the parameters of a parent SPN are learnable transformations of the a-posterior mixing probabilities of its children's sum units. Due to weight sharing and the tree-shaped computation graphs of GSPNs, we obtain the efficiency and efficacy of deep graph networks with the additional advantages of a probabilistic model. We show the model's competitiveness on scarce supervision scenarios, under missing data, and for graph classification in comparison to popular neural models. We complement the experiments with qualitative analyses on hyper-parameters and the model's ability to answer probabilistic queries.

Abstract

Neural Operators are a recent class of data-driven models for learning solutions to Partial Differential Equations (PDEs). Traditionally, these models are trained in an autoregressive fashion using data collected at discrete time points in the evolution of the PDE. This setup gives rise to two problems: (i) poor temporal generalization due to error accumulation and (ii) poor zero-shot super-resolution capabilities. To address these issues, we propose Vectorized Conditional Neural Fields (VCNeF), a general framework that utilizes transformers and implicit neural representations to efficiently solve time-dependent PDEs of varying coefficients. A comprehensive evaluation of VCNeF on the challenging 1D and 2D PDEs from PDEBench demonstrates the superiority of our model over four state-of-the-art baselines. Furthermore, our proposed model achieves faster inference and generalizes better to unseen PDE parameters than the compared models.

Abstract

Diffusion models are the current state of the art for generating photorealistic images. However, controlling the sampling process for constrained image generation tasks such as inpainting remains challenging since exact conditioning on such constraints is intractable. While existing methods use various techniques to approximate the constrained posterior, this paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior, and to leverage this signal to steer the denoising process of diffusion models. Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs). Building upon prior advances, we further scale up PCs and make them capable of guiding the image generation process of diffusion models. Empirical results suggest that our approach can consistently improve the overall quality and semantic coherence of in painted images across three natural image datasets (i.e., CelebA-HQ, ImageNet, and LSUN) with only ~10% additional computational overhead brought by the TPM.

Abstract

Message-passing graph neural networks (MPNNs) emerged as powerful tools for processing graph-structured input. However, they operate on a fixed input graph structure, ignoring potential noise and missing information. Furthermore, their local aggregation mechanism can lead to problems such as over-squashing and limited expressive power in capturing relevant graph structures. Existing solutions to these challenges have primarily relied on heuristic methods, often disregarding the underlying data distribution. Hence, devising principled approaches for learning to infer graph structures relevant to the given prediction task remains an open challenge. In this work, leveraging recent progress in exact and differentiable k-subset sampling, we devise probabilistically rewired MPNNs (PR-MPNNs), which learn to add relevant edges while omitting less beneficial ones. For the first time, our theoretical analysis explores how PR-MPNNs enhance expressive power, and we identify precise conditions under which they outperform purely randomized approaches. Empirically, we demonstrate that our approach effectively mitigates issues like over-squashing and under-reaching. In addition, on established real-world datasets, our method exhibits competitive or superior predictive performance compared to traditional MPNN models and recent graph transformer architectures.

Abstract

SPARQL CONSTRUCT queries allow for the specification of data processing pipelines that transform given input graphs into new output graphs. It is now common to constrain graphs through SHACL shapes allowing users to understand which data they can expect and which not. However, it becomes challenging to understand what graph data can be expected at the end of a data processing pipeline without knowing the particular input data: Shape constraints on the input graph may affect the output graph, but may no longer apply literally, and new shapes may be imposed by the query template. In this paper, we study the derivation of shape constraints that hold on all possible output graphs of a given SPARQL CONSTRUCT query. We assume that the SPARQL CONSTRUCT query is fixed, e.g., being part of a program, whereas the input graphs adhere to input shape constraints but may otherwise vary over time and, thus, are mostly unknown. We study a fragment of SPARQL CONSTRUCT queries (SCCQ) and a fragment of SHACL (Simple SHACL). We formally define the problem of deriving the most restrictive set of Simple SHACL shapes that constrain the results from evaluating a SCCQ over any input graph restricted by a given set of Simple SHACL shapes. We propose and implement an algorithm that statically analyses input SHACL shapes and CONSTRUCT queries and prove its soundness and complexity.

Abstract

Model compression has been an active research field that has been used to reduce the size and complexity of the model. In a recent noteworthy study, ToMe and its variants utilize the Bipartite Soft Matching (BSM) algorithm in which tokens representing patches in an image are split into two sets, and top-k similar tokens from one set are merged. This approach utilizes pre-trained weights, enhances speed, and reduces memory usage. However, these algorithms have some drawbacks. First, the choice of a token-splitting strategy significantly influences algorithm performance since tokens in one set can only perceive tokens in the other set, leading to mis-merging issues. Furthermore, although ToMe is effective in the initial layers, it becomes increasingly problematic in deeper layers as the number of tokens diminishes because of damaged informative tokens. To address these limitations, rather than relying on specific splitting strategies like BSM, we propose a new algorithm called PiToMe. Specifically, we prioritize the protection of informative tokens using an additional factor called energy score. In experiments, PiToMe achieved up to a 50% memory reduction while exhibiting superior off-the-shelf performance on image classification ( keeping a 1.71% average performance drop compared to 2.6% for ToMe) and image-text retrieval (1.35% average performance drop compared to 6.89% for ToMe) compared to previous BSM-based approaches dependent solely on token similarity.

Abstract

The popularity of Knowledge Graphs (KGs) both in industry and academia owes credit to their flexible data model, suitable for data integration from multiple sources. Several KG-based applications such as trust assessment or view maintenance on dynamic data rely on the ability to compute provenance explanations for query results. The how-provenance of a query result is an expression that encodes the records (triples or facts) that explain its inclusion in the result set. This article proposes NPCS, a Native Provenance Computation approach for SPARQL queries. NPCS annotates query results with their how-provenance. By building upon spm-provenance semirings, NPCS supports both monotonic and non-monotonic SPARQL queries. Thanks to its reliance on query rewriting techniques, the approach is directly applicable to already deployed SPARQL engines using different reification schemes -- including RDF*. Our experimental evaluation on two popular SPARQL engines (GraphDB and Stardog) shows that our novel query rewriting brings a significant runtime improvement over existing query rewriting solutions, scaling to RDF graphs with billions of triples.

Abstract

Code for the implementation and benchmark of NPCS, a Native Provenance Computation for SPARQL.The code in this dataset includes the implementation of the NPCS system, which is a middleware for SPARQL endpoints that rewrites queries to queries that annotate answers with provenance polynomials (i.e., how-provenance data). The translation rules implemented for the query rewriting can be seen in the paper.Also, the code contains scripts that include scripts and services to automatize the query execution.We use GraphDB (version 10.2.0) and Stardog (version 9.1.0) for the SPARQL endpoints. Because of the license restrictions, these software products cannot be included in this dataset and must be downloaded from the respective vendors. Also, the data must be loaded using the respective bulk loaders of GraphDB and Stardog.The datasets used in the experiments can be generated synthetic dataset generator of the WatDiv benchmark. The Wikidata dataset corresponds to the full RDF dump from May 22, 2023.Do not hesitate to contact the authors for any inquiries.

Abstract

Knowledge Graphs (KGs) are fundamental for organizing and representing large amounts of information, but they often suffer from incompleteness. Link prediction using Knowledge Graph Embedding (KGE) methods has emerged as a solution to this problem. Many different methods have been proposed to perform link prediction, some of which are a combination of different methods. However, existing approaches that combine different methods typically train models on the entire graph, lacking the diversity seen in machine learning ensembles such as bagging and random forests. In this thesis, we present the novel ensemble approaches UnifEnt and UnifFeat, that divide the KG into sub-graphs by taking advantage of the core principles of bagging and random forests. We evaluated our approach on common KG datasets and showed the benefits of using our method by comparing it to common KGE baseline methods, as well as related work in the area of ensemble methods for link prediction.

Abstract

This work considers the problem of sampling from a probability distribution known up to a normalization constant while satisfying a set of statistical constraints specified by the expected values of general nonlinear functions. This problem finds applications in, e.g., Bayesian inference, where it can constrain moments to evaluate counterfactual scenarios or enforce desiderata such as prediction fairness. Methods developed to handle support constraints, such as those based on mirror maps, barriers, and penalties, are not suited for this task. This work therefore relies on gradient descent-ascent dynamics in Wasserstein space to put forward a discrete-time primal-dual Langevin Monte Carlo algorithm (PD-LMC) that simultaneously constrains the target distribution and samples from it. We analyze the convergence of PD-LMC under standard assumptions on the target distribution and constraints, namely (strong) convexity and log-Sobolev inequalities. To do so, we bring classical optimization arguments for saddle-point algorithms to the geometry of Wasserstein space. We illustrate the relevance and effectiveness of PD-LMC in several applications.

Abstract

Medical narratives are fundamental to the correct identification of a patient’s health condition. This is not only because it describes the patient’s situation. It also contains relevant information about the patient’s context and health state evolution. Narratives are usually vague and cannot be categorized easily. On the other hand, once the patient’s situation is correctly identified based on a narrative, it is then possible to map the patient’s situation into precise classification schemas and ontologies that are machine-readable. To this end, language models can be trained to read and extract elements from these narratives. However, the main problem is the lack of data for model identification and model training in languages other than English. First, gold standard annotations are usually not available due to the high level of data protection for patient data. Second, gold standard annotations (if available) are difficult to access. Alternative available data, like MIMIC (Sci Data 3:1, 2016) is written in English and for specific patient conditions like intensive care. Thus, when model training is required for other types of patients, like oncology (and not intensive care), this could lead to bias. To facilitate clinical narrative model training, a method for creating high-quality synthetic narratives is needed.

Abstract

Transformer models are increasingly used for solving Partial Differential Equations (PDEs). Several adaptations have been proposed, all of which suffer from the typical problems of Transformers, such as quadratic memory and time complexity. Furthermore, all prevalent architectures for PDE solving lack at least one of several desirable properties of an ideal surrogate model, such as (i) generalization to PDE parameters not seen during training, (ii) spatial and temporal zero-shot super-resolution, (iii) continuous temporal extrapolation, (iv) support for 1D, 2D, and 3D PDEs, and (v) efficient inference for longer temporal rollouts. To address these limitations, we propose Vectorized Conditional Neural Fields (VCNeFs), which represent the solution of time-dependent PDEs as neural fields. Contrary to prior methods, however, VCNeFs compute, for a set of multiple spatio-temporal query points, their solutions in parallel and model their dependencies through attention mechanisms. Moreover, VCNeF can condition the neural field on both the initial conditions and the parameters of the PDEs. An extensive set of experiments demonstrates that VCNeFs are competitive with and often outperform existing ML-based surrogate models.

Abstract

This CNVVE Dataset contains raw audio samples encompassing six distinct classes of voice expressions, namely “Uh-huh” or “mm-hmm”, “Uh-uh” or“mm-mm”, “Hush” or “Shh”, “Psst”, “Ahem”, and Continuous humming, e.g., “hmmm.” Audio samples of each class are found in the respective folders. The samples are recorded through a dedicated website for data collection that defines the purpose and type of voice data by providing example recordings to participants as well as the expressions’ written equivalent, e.g., “Uh-huh”. Audio recordings were automatically saved in the .wav format and kept anonymous, with a sampling rate of 48 kHz and a bit depth of 32 bits.This dataset contains a raw version of the samples. A cleaned version of these samples can be found on https://doi.org/10.18419/darus-3898. For more info, please check the paper or feel free to contact the authors for any inquiries.

Abstract

This CNVVE Dataset contains clean audio samples encompassing six distinct classes of voice expressions, namely “Uh-huh” or “mm-hmm”, “Uh-uh” or“mm-mm”, “Hush” or “Shh”, “Psst”, “Ahem”, and Continuous humming, e.g., “hmmm.” Audio samples of each class are found in the respective folders. These audio samples have undergone a thorough cleaning process. The raw samples are published in https://doi.org/10.18419/darus-3897. Initially, we applied the Google WebRTC voice activity detection (VAD) algorithm on the given audio files to remove noise or silence from the collected voice signals. The intensity was set to "2", which could be a value between "1" and "3". However, because of variations in the data, some files required additional manual cleaning. These outliers, characterized by sharp click sounds (such as those occurring at the end of recordings), were addressed. The samples are recorded through a dedicated website for data collection that defines the purpose and type of voice data by providing example recordings toparticipants as well as the expressions’ written equivalent, e.g., “Uh-huh”. Audio recordings were automatically saved in the .wav format and keptanonymous, with a sampling rate of 48 kHz and a bit depth of 32 bits. For more info, please check the paper or feel free to contact the authors for any inquiries.

Abstract

Social groups are central to political discussions. However, detecting social groups in text often relies on pre-determined socio-demographic categories or supervised learning methods that require extensive hand-labeled datasets. In this paper, we propose a methodology designed to leverage the potential of Large Language Models (LLMs) for the identification and annotation of social groups in text. The experiments show that open LLMs like Llama-2-70B-Chat and Mixtral-8-7B can reliably be used to annotate social groups in a few-shot scenario without the need for supervised learning. The automatically obtained annotations largely match human annotations on random samples from the Reddit Politosphere, resulting in micro-F1 scores of 0.71 and 0.83, respectively.

Abstract

A popular paradigm for offline Reinforcement Learning (RL) tasks is to first fit the offline trajectories to a sequence model, and then prompt the model for actions that lead to high expected return. In addition to obtaining accurate sequence models, this paper highlights that tractability, the ability to exactly and efficiently answer various probabilistic queries, plays an important role in offline RL. Specifically, due to the fundamental stochasticity from the offline data-collection policies and the environment dynamics, highly non-trivial conditional/constrained generation is required to elicit rewarding actions. it is still possible to approximate such queries, we observe that such crude estimates significantly undermine the benefits brought by expressive sequence models. To overcome this problem, this paper proposes Trifle (Tractable Inference for Offline RL), which leverages modern Tractable Probabilistic Models (TPMs) to bridge the gap between good sequence models and high expected returns at evaluation time. Empirically, Trifle achieves the most state-of-the-art scores in 9 Gym-MuJoCo benchmarks against strong baselines. Further, owing to its tractability, Trifle significantly outperforms prior approaches in stochastic environments and safe RL tasks (e.g. with action constraints) with minimum algorithmic modifications.

Abstract

In this work, we propose a simple transformer-based baseline for multimodal molecular representation learning, integrating three distinct modalities: SMILES strings, 2D graph representations, and 3D conformers of molecules. A key aspect of our approach is the aggregation of 3D conformers, allowing the model to account for the fact that molecules can adopt multiple conformations-an important factor for accurate molecular representation. The tokens for each modality are extracted using modality-specific encoders: a transformer for SMILES strings, a message-passing neural network for 2D graphs, and an equivariant neural network for 3D conformers. The flexibility and modularity of this framework enable easy adaptation and replacement of these encoders, making the model highly versatile for different molecular tasks. The extracted tokens are then combined into a unified multimodal sequence, which is processed by a downstream transformer for prediction tasks. To efficiently scale our model for large multimodal datasets, we utilize Flash Attention 2 and bfloat16 precision. Despite its simplicity, our approach achieves state-of-the-art results across multiple datasets, demonstrating its effectiveness as a strong baseline for multimodal molecular representation learning.

Abstract

Liberal political philosophy advocates for the policy of equal treatment as blindness, which seeks to achieve fairness by treating individuals without considering their protected characteristics directly. However, this policy has faced longstanding criticism for perpetuating existing inequalities. In machine learning, this policy can be translated into the concept of fairness as unawareness, and be measured using disparate treatment metrics such as Demographic Parity (a.k.a. Statistical Parity). Our analysis reveals that Demographic Parity does not faithfully measure whether individuals are being treated independently of the protected attribute by the model. We introduce the Explanation Disparity metric to measure fairness under equal treatment as blindness policy. Our metric evaluates the fairness of predictive models by analyzing the extent to which the protected attribute can be inferred from the distribution of explanation values, specifically using Shapley values. The proposed metric tests for statistical independence of the explanation distributions over populations with different protected characteristics. We show the theoretical properties of "Explanation Disparity" and devise an equal treatment inspector based on the AUC of a Classifier Two-Sample Test. We experiment with synthetic and natural data to demonstrate and compare the notion with related ones.

Abstract

Solving partial differential equations (PDEs) is a fundamental problem in engineering and science. While neural PDE solvers can be more efficient than established numerical solvers, they often require large amounts of training data that is costly to obtain. Active Learning (AL) could help surrogate models reach the same accuracy with smaller training sets by querying classical solvers with more informative initial conditions and PDE parameters. While AL is more common in other domains, it has yet to be studied extensively for neural PDE solvers. To bridge this gap, we introduce AL4PDE, a modular and extensible active learning benchmark. It provides multiple parametric PDEs and state-of-the-art surrogate models for the solver-in-the-loop setting, enabling the evaluation of existing and the development of new AL methods for PDE solving. We use the benchmark to evaluate batch active learning algorithms such as uncertainty- and feature-based methods. We show that AL reduces the average error by up to 71% compared to random sampling and significantly reduces worst-case errors. Moreover, AL generates similar datasets across repeated runs, with consistent distributions over the PDE parameters and initial conditions. The acquired datasets are reusable, providing benefits for surrogate models not involved in the data generation.

Abstract

Prompt learning methods are gaining increasing attention due to their ability to customize large vision-language models to new domains using pre-trained contextual knowledge and minimal training data. However, existing works typically rely on optimizing unified prompt inputs, often struggling with fine-grained classification tasks due to insufficient discriminative attributes. To tackle this, we consider a new framework based on a dual context of both domain-shared and class-specific contexts, where the latter is generated by Large Language Models (LLMs) such as GPTs. Such dual prompt methods enhance the model's feature representation by joining implicit and explicit factors encoded in LLM knowledge. Moreover, we formulate the Unbalanced Optimal Transport (UOT) theory to quantify the relationships between constructed prompts and visual tokens. Through partial matching, UOT can properly align discrete sets of visual tokens and prompt embeddings under different mass distributions, which is particularly valuable for handling irrelevant or noisy elements, ensuring that the preservation of mass does not restrict transport solutions. Furthermore, UOT's characteristics integrate seamlessly with image augmentation, expanding the training sample pool while maintaining a reasonable distance between perturbed images and prompt inputs. Extensive experiments across few-shot classification and adapter settings substantiate the superiority of our model over current state-of-the-art baselines.

Abstract

A molecule's 2D representation consists of its atoms, their attributes, and the molecule's covalent bonds. A 3D (geometric) representation of a molecule is called a conformer and consists of its atom types and Cartesian coordinates. Every conformer has a potential energy, and the lower this energy, the more likely it occurs in nature. Most existing machine learning methods for molecular property prediction consider either 2D molecular graphs or 3D conformer structure representations in isolation. Inspired by recent work on using ensembles of conformers in conjunction with 2D graph representations, we propose E(3)-invariant molecular conformer aggregation networks. The method integrates a molecule's 2D representation with that of multiple of its conformers. Contrary to prior work, we propose a novel 2D-3D aggregation mechanism based on a differentiable solver for the Fused Gromov-Wasserstein Barycenter problem and the use of an efficient conformer generation method based on distance geometry. We show that the proposed aggregation mechanism is E(3) invariant and propose an efficient GPU implementation. Moreover, we demonstrate that the aggregation mechanism helps to significantly outperform state-of-the-art molecule property prediction methods on established datasets.

Abstract

Temporal knowledge graphs represent temporal facts (s,p,o,τ) relating a subject s and an object o via a relation label p at time τ, where τ could be a time point or time interval. Temporal knowledge graphs may exhibit static temporal patterns at distinct points in time and dynamic temporal patterns between different timestamps. In order to learn a rich set of static and dynamic temporal patterns and apply them for inference, several embedding approaches have been suggested in the literature. However, as most of them resort to single underlying embedding spaces, their capability to model all kinds of temporal patterns was severely limited by having to adhere to the geometric property of their one embedding space. We lift this limitation by an embedding approach that maps temporal facts into a product space of several heterogeneous geometric subspaces with distinct geometric properties, i.e.\ Complex, Dual, and Split-complex spaces. In addition, we propose a temporal-geometric attention mechanism to integrate information from different geometric subspaces conveniently according to the captured relational and temporal information. Experimental results on standard temporal benchmark datasets favorably evaluate our approach against state-of-the-art models.

Abstract

In recent years, large-scale knowledge graphs have emerged, integrating data from various sources. Often, this data includes assertions about other assertions, establishing contexts in which these assertions hold. A recent enhancement to RDF, known as RDF-star, allows for statements about statements and is currently under consideration as a W3C standard. However, RDF-star lacks a defined semantics for such statements and lacks intrinsic mechanisms to operate on them. This thesis describes and implements a novel query language, termed eSPARQL, tailored for epistemic RDF-star metadata and grounded in four-valued logic. Our language builds on SPARQL-star, the query language for RDF-star, by incorporating an expanded FROM clause, called FROM BELIEF, designed to manage multiple, and occasionally conflicting, beliefs. eSPARQL’s capabilities are demonstrated through four example queries, showcasing its ability to (i) retrieve individual beliefs, (ii) aggregate beliefs, (iii) identify conflicts between individuals, and (iv) handle nested beliefs (beliefs about beliefs). The implementation of eSPARQL developed in this thesis is built on top of an existing SPARQL-star query engine. In this implementation, the execution process of a eSPARQL consists of two phases. First, the expression in the FROM BELIEF clause, called belief query, is translated into a SPARQL-star CONSTRUCT query that generates an intermediary graph, containing the beliefs of the subjects described in the belief query. In the second phase, This intermediary graph is then processed with the graph pattern of the eSPARQL by translating it to a graph pattern that can be processed by a standard SPARQ-star engine. In this last phase, the implementation translates eSPARQL operations to SPARQL-star, and checks if the pattern contains nested eSPARQL queries to be processed recursively. We study two research questions: (RQ1) Does the eSPARQL implementation scale? and (RQ2) How the eSPARQL implementation execution times compare with the execution time of manually written SPARQL-star queries? To answer these research questions, use the four example eSPARQL queries that showcase the abilities of eSPARQL and create a synthetic dataset generator that generates graphs of multiple sizes. Additionally, for research question RQ2, we manually generate SPARQL-star queries that are equivalent to the example eSPARQL queries. Regarding research question RQ1, our results show that eSPARQL has an execution time that is proportional with the data size. Regarding research question RQ2, except for one question, the manually written SPARQL-star queries are clearly faster than our implementation. Although the implementation showed to be slower than the manually generated SPARQL-star queries, the eSPARQL queries are shorter and easier to understand. This positive aspect of eSPARQL, can motivate further studies on how to optimize the eSPARQL implementation.

Abstract

In Open-Set Domain Generalization (OSDG), the model is exposed to both new variations of data appearance (domains) and open-set conditions, where both known and novel categories are present at test time. The challenges of this task arise from the dual need to generalize across diverse domains and accurately quantify category novelty, which is critical for applications in dynamic environments. Recently, meta-learning techniques have demonstrated superior results in OSDG, effectively orchestrating the meta-train and -test tasks by employing varied random categories and predefined domain partition strategies. These approaches prioritize a well-designed training schedule over traditional methods that focus primarily on data augmentation and the enhancement of discriminative feature learning. The prevailing meta-learning models in OSDG typically utilize a predefined sequential domain scheduler to structure data partitions. However, a crucial aspect that remains inadequately explored is the influence brought by strategies of domain schedulers during training. In this paper, we observe that an adaptive domain scheduler benefits more in OSDG compared with prefixed sequential and random domain schedulers. We propose the Evidential Bi-Level Hardest Domain Scheduler (EBiL-HaDS) to achieve an adaptive domain scheduler. This method strategically sequences domains by assessing their reliabilities in utilizing a follower network, trained with confidence scores learned in an evidential manner, regularized by max rebiasing discrepancy, and optimized in a bi-level manner. The results show that our method substantially improves OSDG performance and achieves more discriminative embeddings for both the seen and unseen categories.

Abstract

Large-language models (LLMs) have the potential to support a wide range of applications like conversational agents, creative writing, text improvement, and general query answering. However, they are ill-suited for query answering in high-stake domains like medicine because they generate answers at random and their answers are typically not robust - even the same query can result in different answers when prompted multiple times. In order to improve the robustness of LLM queries, we propose using ranking queries repeatedly and to aggregate the queries using methods from social choice theory. We study ranking queries in diagnostic settings like medical and fault diagnosis and discuss how the Partial Borda Choice function from the literature can be applied to merge multiple query results. We discuss some additional interesting properties in our setting and evaluate the robustness of our approach empirically.

Abstract

Message-passing graph neural networks (MPNNs) have emerged as a powerful paradigm for graph-based machine learning. Despite their effectiveness, MPNNs face challenges such as under-reaching and over-squashing, where limited receptive fields and structural bottlenecks hinder information flow in the graph. While graph transformers hold promise in addressing these issues, their scalability is limited due to quadratic complexity regarding the number of nodes, rendering them impractical for larger graphs. Here, we propose implicitly rewired message-passing neural networks (IPR-MPNNs), a novel approach that integrates implicit probabilistic graph rewiring into MPNNs. By introducing a small number of virtual nodes, i.e., adding additional nodes to a given graph and connecting them to existing nodes, in a differentiable, end-to-end manner, IPR-MPNNs enable long-distance message propagation, circumventing quadratic complexity. Theoretically, we demonstrate that IPR-MPNNs surpass the expressiveness of traditional MPNNs. Empirically, we validate our approach by showcasing its ability to mitigate under-reaching and over-squashing effects, achieving state-of-the-art performance across multiple graph datasets. Notably, IPR-MPNNs outperform graph transformers while maintaining significantly faster computational efficiency.

Abstract

Field surveys to collect data from fluvial ecosystems traditionally focus on specific phenomena related to geomorphology or hydrology. Low-cost unmanned aerial vehicles (UAVs) additionally empower the fast and massive collection of airborne photogrammetry, providing geospatially explicit information. This remote sensing data complements field surveys by offering contextual information on geomorphological conditions, including digital terrain models. AI-based image recognition can augment contextual information to extrapolate archetypal object classes through name labels, such as “gravel”, “sand”, “plant”, or “large wood”. However, obtaining sufficient ground truth data for these classifications, particularly in morphodynamic fluvial environments, is challenging and induces high costs. This study introduces a transfer learning approach to address the challenge of low data availability, enabling AI-based mapping of complex objects in fluvial landscapes. We leverage the learned general structure of a deep convolutional neural network (CNN) pre-trained on a broad range of images. The fixed latent features of the pre-trained CNN stem from GoogLeNet. A fixed feature extractor serves to classify objects with limited data amounts. Satisfactory performance is measured with a recall rate, expressing the ability of a model to find all occurrences of a class on an image. High spatial heterogeneity in the locations of measurements on the x-y plane improves model performance. With a minimum of 400 labeled instances, the model achieves a satisfactory 93.75-% recall for a “large wood” target class, providing evidence of the effectiveness of transfer learning in remote sensing for geomorphological studies. This ability to detect large woods in river environments is critical to restoration efforts as it helps create fish habitat, which is essential to supporting biodiversity.

Abstract

This dataset contains the implementation code for an algorithm to infer SHACL shapes that the graph returned by an SPARQL CONSTRUCT query must satisfy if the input satisfies a given set of SHACL shapes. This dataset also includes an evaluation for the algorithm. The algorithm implemented in this dataset is proposed in the paper From Shapes to Shapes: Inferring SHACL Shapes for Results of SPARQL CONSTRUCT Queries. To execute the code, follow the instructions in the README.md file. For more info, please check the paper, and please have no hesitation to contact the authors for any inquiries.

Abstract

Graph Neural Networks (GNNs) are a popular class of machine learning models. Inspired by the learning to explain (L2X) paradigm, we propose L2XGNN, a framework for explainable GNNs that provides faithful explanations by design. L2XGNN learns a mechanism for selecting explanatory subgraphs (motifs), which are exclusively used in the GNN message-passing operations. L2XGNN can select, for each input graph, a subgraph with specific properties, such as being sparse and connected. Imposing such constraints on the motifs often leads to more interpretable and effective explanations. Experiments on several datasets suggest that L2XGNN achieves the same classification accuracy as baseline methods using the entire input graph while ensuring that only the provided explanations are used to make predictions. Moreover, we show that L2XGNN can identify motifs responsible for the graph's properties it is intended to predict.

Abstract

Physics-informed neural networks (PINNs) have emerged as a promising approach for solving partial differential equations (PDEs) using neural networks, particularly in data-scarce scenarios due to their unsupervised training capability. However, a key limitation is the need for re-optimization with each change in PDE parameters, similar to the challenge in traditional numerical methods where each system of equations corresponds to a specific PDE instance. This characteristic poses a barrier to the widespread adoption of PINNs across scientific and engineering applications. This survey explores research addressing this limitation through transfer learning and meta-learning, synthesizing insights to establish a foundation for efficient data generation strategies tailored to PINNs. These methods can potentially improve PINNs' training efficiency, enabling quicker adaptation to new PDEs with fewer data and computational demands. While numerical methods directly solve systems of equations to derive solutions, neural networks implicitly learn solutions by adjusting their parameters. One notable advantage of neural networks lies in their capacity to abstract away from specific problem domains, enabling them to retain, discard, or adapt learned representations to efficiently address similar problems. By understanding how these techniques can be applied to PINNs, this survey seeks to identify promising directions for future research to enable the widespread adoption of PINNs across a wide range of scientific and engineering applications.

Abstract

Increasing the throughput of the Transformer architecture, a foundational component used in numerous state-of-the-art models for vision and language tasks (e.g., GPT, LLaVa), is an important problem in machine learning. One recent and effective strategy is to merge token representations within Transformer models, aiming to reduce computational and memory requirements while maintaining accuracy. Prior works have proposed algorithms based on Bipartite Soft Matching (BSM), which divides tokens into distinct sets and merges the top k similar tokens. However, these methods have significant drawbacks, such as sensitivity to token-splitting strategies and damage to informative tokens in later layers. This paper presents a novel paradigm called PiToMe, which prioritizes the preservation of informative tokens using an additional metric termed the energy score. This score identifies large clusters of similar tokens as high-energy, indicating potential candidates for merging, while smaller (unique and isolated) clusters are considered as low-energy and preserved. Experimental findings demonstrate that PiToMe saved from 40-60% FLOPs of the base models while exhibiting superior off-the-shelf performance on image classification (0.5% average performance drop of ViT-MAE-H compared to 2.6% as baselines), image-text retrieval (0.3% average performance drop of CLIP on Flickr30k compared to 4.5% as others), and analogously in visual questions answering with LLaVa-7B. Furthermore, PiToMe is theoretically shown to preserve intrinsic spectral properties of the original token space under mild conditions.

Abstract

This paper presents OmniJARVIS, a novel Vision-Language-Action (VLA) model for open-world instruction-following agents in Minecraft. Compared to prior works that either emit textual goals to separate controllers or produce the control command directly, OmniJARVIS seeks a different path to ensure both strong reasoning and efficient decision-making capabilities via unified tokenization of multimodal interaction data. First, we introduce a self-supervised approach to learn a behavior encoder that produces discretized tokens for behavior trajectories and an imitation learning policy decoder conditioned on these tokens. These additional behavior tokens will be augmented to the vocabulary of pretrained Multimodal Language Models. With this encoder, we then pack long-term multimodal interactions involving task instructions, memories, thoughts, observations, textual responses, behavior trajectories, etc into unified token sequences and model them with autoregressive transformers. Thanks to the semantically meaningful behavior tokens, the resulting VLA model, OmniJARVIS, can reason (by producing chain-of-thoughts), plan, answer questions, and act (by producing behavior tokens for the imitation learning policy decoder). OmniJARVIS demonstrates excellent performances on a comprehensive collection of atomic, programmatic, and open-ended tasks in open-world Minecraft. Our analysis further unveils the crucial design principles in interaction data formation, unified tokenization, and its scaling potentials.

Abstract

This is a PyTorch implementation of the paper Hyperbolic Embedding Inference for Structured Multi-Label Prediction published in NeurIPS 2022. The code provides the Python scripts to reproduce the experiments in the paper, as well as a proof-of-concept example of the method. To execute the code, follow the instructions in the README.md file. For more info, please check the paper. Please have no hesitation to contact the authors for any inquiries.

Abstract

This code is a PyTorch implementation of the paper "NestE: Modeling Nested Relational Structures for Knowledge Graph Reasoning (AAAI'24)".NestE is a knowledge graph embedding method that can encode nested facts represented by quoted triples (h,r,t) in which the subject and object are triples themselves, e.g., ((BarackObama, holds_position, President), succeed_by, (DonaldTrump, holds_position, President)).We implement six variant models of NetsE based on different hypercomplex number systems. NestE_Q.py for NestE with quaternion. NestE_H.py for NestE with hyperbolic quaternion. NestE_D.py for NestE with split quaternion. NestE_B.py, NestE_HB.py, and NestE_DB.py are the respective version with a translation component. This code is used to reproduce the experiments of the paper. To execute the code, follow the instructions in the README.md file.

Abstract

This is a Pytorch implementation of the paper Shrinking Embeddings for Hyper-relational Knowledge Graphs published in ACL'23.This code is used to reproduce the experiments of the method ShrinkE, a geometric embedding approach for hyper-relational knowledge graphs. The code is implemented with Python 3 and pytorch. The code is tested on public datasets which can be download from StarE. To execute the code, follow the instructions in the README.md file. For more info, please check the paper or feel free to contact the authors for any inquiries.

Abstract

This is a Pytorch implementation of the paper Ultrahyperbolic Knowledge Graph Embeddings published in KDD 2022. This code is used to reproduce the experiments of the method UltraE, a geometric embedding approach for knowledge graph embeddings. The code is tested on public datasets which can be downloaded from KGEmb. To execute the code, follow the instructions in the README.md file. For more info, please check the paper or feel free to contact the authors for any inquiries.

Abstract

The ability to perform fast and accurate atomistic simulations is crucial for advancing the chemical sciences. By learning from high-quality data, machine-learned interatomic potentials achieve accuracy on par with ab initio and first-principles methods at a fraction of their computational cost. The success of machine-learned interatomic potentials arises from integrating inductive biases such as equivariance to group actions on an atomic system, e.g., equivariance to rotations and reflections. In particular, the field has notably advanced with the emergence of equivariant message passing. Most of these models represent an atomic system using spherical tensors, tensor products of which require complicated numerical coefficients and can be computationally demanding. Cartesian tensors offer a promising alternative, though state-of-the-art methods lack flexibility in message-passing mechanisms, restricting their architectures and expressive power. This work explores higher-rank irreducible Cartesian tensors to address these limitations. We integrate irreducible Cartesian tensor products into message-passing neural networks and prove the equivariance and traceless property of the resulting layers. Through empirical evaluations on various benchmark data sets, we consistently observe on-par or better performance than that of state-of-the-art spherical and Cartesian models.

Abstract

Efficiently creating a concise but comprehensive data set for training machine-learned interatomic potentials (MLIPs) is an under-explored problem. Active learning, which uses biased or unbiased molecular dynamics (MD) to generate candidate pools, aims to address this objective. Existing biased and unbiased MD-simulation methods, however, are prone to miss either rare events or extrapolative regions—areas of the configurational space where unreliable predictions are made. This work demonstrates that MD, when biased by the MLIP’s energy uncertainty, simultaneously captures extrapolative regions and rare events, which is crucial for developing uniformly accurate MLIPs. Furthermore, exploiting automatic differentiation, we enhance bias-forces-driven MD with the concept of bias stress. We employ calibrated gradient-based uncertainties to yield MLIPs with similar or, sometimes, better accuracy than ensemble-based methods at a lower computational cost. Finally, we apply uncertainty-biased MD to alanine dipeptide and MIL-53(Al), generating MLIPs that represent both configurational spaces more accurately than models trained with conventional MD.

Abstract

Knowledge graph embedding (KGE) models are often used to predict missing links for knowledge graphs (KGs). However, multiple KG embeddings can perform almost equally well for link prediction yet give conflicting predictions for unseen queries. This phenomenon is termed predictive multiplicity in the literature. It poses substantial risks for KGE-based applications in high-stake domains but has been overlooked in KGE research. We define predictive multiplicity in link prediction, introduce evaluation metrics and measure predictive multiplicity for representative KGE methods on commonly used benchmark datasets. Our empirical study reveals significant predictive multiplicity in link prediction, with 8% to 39% testing queries exhibiting conflicting predictions. We address this issue by leveraging voting methods from social choice theory, significantly mitigating conflicts by 66% to 78% in our experiments.

Abstract

Multistep prediction models are essential for the simulation and model-predictive control of dynamical systems. Verifying the safety of such models is a multi-faceted problem requiring both system-theoretic guarantees as well as establishing trust with human users. In this work, we propose a novel approach, ReLiNet (Recurrent Linear Parameter Varying Network), to ensure safety for multistep prediction of dynamical systems. Our approach simplifies a recurrent neural network to a switched linear system that is constrained to guarantee exponential stability, which acts as a surrogate for safety from a system-theoretic perspective. Furthermore, ReLiNet’s computation can be reduced to a single linear model for each time step, resulting in predictions that are explainable by definition, thereby establishing trust from a human-centric perspective. Our quantitative experiments show that ReLiNet achieves prediction accuracy comparable to that of state-of-the-art recurrent neural networks, while achieving more faithful and robust explanations compared to the model-agnostic explanation method of LIME.

Abstract

Non-verbal voice expressions (NVVEs) have been adopted as a means of human-computer interaction in research studies. However, exploring non-verbal voice-based interactions has been constrained by the limited availability of suitable training data and computational methods for classifying such expressions, leading to a focus on simple binary inputs. We address this issue with a new dataset containing 950 audio samples comprising 6 classes of voice expressions. The data were collected from 42 speakers who donated voice recordings. The classifier was trained on the data using features derived from mel-spectrograms. Furthermore, we studied the effectiveness of data augmentation and improved over the baseline model accuracy significantly with a test accuracy of 96.6% in a 5-fold cross-validation. We have made CNVVE publicly accessible in the hope that it will serve as a benchmark for future research.

Abstract

Diffusion models are the current state of the art for generating photorealistic images. However, controlling the sampling process for constrained image generation tasks such as inpainting remains challenging since exact conditioning on such constraints is intractable. While existing methods use various techniques to approximate the constrained posterior, this paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior, and to leverage this signal to steer the denoising process of diffusion models. Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs). Building upon prior advances, we further scale up PCs and make them capable of guiding the image generation process of diffusion models. Empirical results suggest that our approach can consistently improve the overall quality and semantic coherence of in painted images across three natural image datasets (i.e., CelebA-HQ, ImageNet, and LSUN) with only ~10% additional computational overhead brought by the TPM.

Abstract

This repository contains the necessary scripts to reproduce the results from our paper "ReLiNet: Stable and Explainable Multistep Prediction with Recurrent Linear Parameter Varying Networks". See the README file for more information. The most current version of this software is available on Github.

Abstract

deepsysid is a system identification toolkit for multistep prediction using deep learning and hybrid methods. The toolkit is easy to use. After you follow the instructions in the README, you will be able to download a dataset, run hyperparameter optimization and identify your best-performing multistep prediction models with just three commands. The most current version of this software is available on GitHub.

Abstract

The integration of data from various disciplines, including requirements and regulations, is essential in the co-design of buildings. Constraints that arise during design, fabrication, or construction are mainly considered in the later stages of the design process. This often leads to costly revisions during fabrication or construction. While there are some works proposing Semantic Web approaches or ad-hoc methods to identify constraint violations in the early stages of design, they mainly rely on data derived from monolithic data schemas. This paper presents a Semantic Web approach that validates and checks building data derived from federated data schemas. The proposed approach is exemplified through the design process of a segmented timber shell, a complex structure requiring negotiations across architectural, engineering, and fabrication parameters. The Building Habitat object Models (BHoM) framework is used to represent federated building data, and the approach enables a seamless movement between the graph environment and object models to support the design process. Additionally, the study discusses various technologies that aid in knowledge inference or data validation. The results of this study suggest that the use of Semantic Web technologies in conjunction with federated data schemas has significant potential to enhance co-design processes in the building industry. While further research is needed to assess its effectiveness in other scenarios, our application of this approach in checking and validating building data for a complex segmented timber shell structure demonstrates its promise as a means to streamline the design process.

Abstract

Knowledge graphs (KGs) are vast collections of machine-readable information, usually modeled in RDF and queried with SPARQL. KGs have opened the door to a plethora of applications such as Web search or smart assistants that query and process the knowledge contained in those KGs. An important, but often disregarded, aspect of querying KGs is query provenance: explanations of the data sources and transformations that made a query result possible. In this article we demonstrate, through a Web application, the capabilities of SPARQLprov, an engine-agnostic method that annotates query results with how-provenance annotations. To this end, SPARQLprov resorts to query rewriting techniques, which make it applicable to already deployed SPARQL endpoints. We describe the principles behind SPARQLprov and discuss perspectives on visualizing how-provenance explanations for SPARQL queries.

Abstract

Predicting missing links between entities in a knowledge graph is a fundamental task to deal with the incompleteness of data on the Web. Knowledge graph embeddings map nodes into a vector space to predict new links, scoring them according to geometric criteria. Relations in the graph may follow patterns that can be learned, e.g., some relations might be symmetric and others might be hierarchical. However, the learning capability of different embedding models varies for each pattern and, so far, no single model can learn all patterns equally well. In this paper, we combine the query representations from several models in a unified one to incorporate patterns that are independently captured by each model. Our combination uses attention to select the most suitable model to answer each query. The models are also mapped onto a non-Euclidean manifold, the Poincaré ball, to capture structural patterns, such as hierarchies, besides relational patterns, such as symmetry. We prove that our combination provides a higher expressiveness and inference power than each model on its own. As a result, the combined model can learn relational and structural patterns. We conduct extensive experimental analysis with various link prediction benchmarks showing that the combined model outperforms individual models, including state-of-the-art approaches.

Abstract

Logical patterns, such as symmetry and composition, have been proven to be beneficial in the knowledge graph completion task. However, their influence has been unexplored in first-order logical (FOL) query reasoning methods. In this work, we present an inductive bias for query embedding models, Patternaware Cone Embedding (PConE), to support learning and reasoning with logical patterns. PConE combines the advantages of cones and the rotation operator for powerful algebraic operations for pattern inference. Our experiments demonstrate how the capability to capture logical patterns positively impacts the results of query answering.

Abstract

Abstract Established reduced-order modeling (ROM) methods, for instance, Galerkin-projection, approximate the solution by linearly projecting high-dimensional spaces to a lower-dimensional space spanned by the reduced basis. However, the accuracy of these methods may be insufficient for complex and multiscale simulations due to the restriction to a linear space. Alternatively, autoencoders (AEs) can be used for nonlinear dimensionality reduction. We combine nonlinear dimensionality reduction techniques with time series prediction to build data-driven ROMs. The presented framework consists of two-level neural networks. The high-dimensional space is nonlinearly compressed in the first level using the encoder function of the AE. Subsequently, the temporal evolution of the latent vector is predicted in the second level. The original solution can be easily reconstructed using the decoder function of the AE. In comparison with the projection-based ROMs, for example, proper orthogonal decomposition (POD) with Galerkin projection, this framework allows naturally to include parameters in the prediction without nonlinear interpolation of the linear basis. We demonstrate the framework on a two-dimensional flow field simulation around circular bodies parameterized with the inlet fluid velocity.

Abstract

Understanding traffic scenes requires considering heterogeneous information about dynamic agents and the static infrastructure. In this work we propose SCENE, a methodology to encode diverse traffic scenes in heterogeneous graphs and to reason about these graphs using a heterogeneous Graph Neural Network encoder and task-specific decoders. The heterogeneous graphs, whose structures are defined by an ontology, consist of different nodes with type-specific node features and different relations with type-specific edge features. In order to exploit all the information given by these graphs, we propose to use cascaded layers of graph convolution. The result is an encoding of the scene. Task-specific decoders can be applied to predict desired attributes of the scene. Extensive evaluation on two diverse binary node classification tasks show the main strength of this methodology: despite being generic, it even manages to outperform task-specific baselines. The further application of our methodology to the task of node classification in various knowledge graphs shows its transferability to other domains.

Abstract

Large-language models (LLMs) have the potential to support a wide range of applications like conversational agents, creative writing, text improvement, and general query answering. However, they are ill-suited for query answering in high-stake domains like medicine because they generate answers at random and their answers are typically not robust - even the same query can result in different answers when prompted multiple times. In order to improve the robustness of LLM queries, we propose using ranking queries repeatedly and to aggregate the queries using methods from social choice theory. We study ranking queries in diagnostic settings like medical and fault diagnosis and discuss how the Partial Borda Choice function from the literature can be applied to merge multiple query results. We discuss some additional interesting properties in our setting and evaluate the robustness of our approach empirically.

Abstract

The Description Logic (DL) EL is a lightweight DL that has a favorable trade-off between expressive power and reasoning complexity and has been widely used in many real-world applications. Statistical EL (SEL) extends EL by allowing conditional probabilities over axioms. Unlike other probabilistic DLs, the probabilistic semantics of SEL is statistical, meaning that probabilities express proportions in a population rather than subjective beliefs. One major challenge is that reasoning in SEL is ExpTimecomplete. To overcome this problem, we propose to use embeddings to perform approximate inference over SEL ontologies. This poster paper demonstrates the progress of the ongoing research, showcasing a demonstration through a simplified example, providing preliminary findings, and outlining the future work.

Abstract

The paucity of labeled data is a typical challenge in the automotive industry. Annotating time-series measurements requires solid domain knowledge and in-depth exploratory data analysis, which implies a high labeling effort. Conventional Active Learning (AL) addresses this issue by actively querying the most informative instances based on the estimated classification probability and retraining the model iteratively. However, the learning efficiency strongly relies on the initial model, resulting in the trade-of between the size of the initial dataset and the query number. This paper proposes a novel Few-Shot Learning (FSL)-based AL framework, which addresses the trade-of problem by incorporating a Prototypical Network (ProtoNet) in the AL iterations. The results show an improvement, on the one hand, in the robustness to the initial model and, on the other hand, in the learning efficiency of the ProtoNet through the active selection of the support set in each iteration. This framework was validated on UCI HAR/HAPT dataset and a real-world braking maneuver dataset. The learning performance significantly surpasses traditional AL algorithms on both datasets, achieving 90% classification accuracy with 10% and 5% labeling effort, respectively.

Abstract

One of the main drivers of the recent advances in authorship verification is the PAN large-scale authorship dataset. Despite generating significant progress in the field, inconsistent performance differences between the closed and open test sets have been reported. To this end, we improve the experimental setup by proposing five new public splits over the PAN dataset, specifically designed to isolate and identify biases related to the text topic and to the author's writing style. We evaluate several BERT-like baselines on these splits, showing that such models are competitive with authorship verification state-of-the-art methods. Furthermore, using explainable AI, we find that these baselines are biased towards named entities. We show that models trained without the named entities obtain better results and generalize better when tested on DarkReddit, our new dataset for authorship verification.

Abstract

Recent knowledge graph (KG) embeddings have been advanced by hyperbolic geometry due to its superior capability for representing hierarchies. The topological structures of real-world KGs, however, are rather heterogeneous, i.e., a KG is composed of multiple distinct hierarchies and non-hierarchical graph structures. Therefore, a homogeneous (either Euclidean or hyperbolic) geometry is not sufficient for fairly representing such heterogeneous structures. To capture the topological heterogeneity of KGs, we present an ultrahyperbolic KG embedding (UltraE) in an ultrahyperbolic (or pseudo-Riemannian) manifold that seamlessly interleaves hyperbolic and spherical manifolds. In particular, we model each relation as a pseudo-orthogonal transformation that preserves the pseudo-Riemannian bilinear form. The pseudo-orthogonal transformation is decomposed into various operators (i.e., circular rotations, reflections and hyperbolic rotations), allowing for simultaneously modeling heterogeneous structures as well as complex relational patterns. Experimental results on three standard KGs show that UltraE outperforms previous Euclidean, hyperbolic, and mixed-curvature KG embedding approaches.

Abstract

This dataset contains data of 125 1-hour simulations of ship motion during various sea states performing random maneuvers in 4 degrees of freedom (surge-sway-yaw-roll). The original ship is a patrol ship developed by Perez et al. 1. We have extended it with a set of two symmetrically placed rudder propellers. Additionally, we simulate wind forces according to Isherwood's wind model 2. Wind-induced waves are generated with the JONSWAP spectrum 3 and the corresponding wave forces are then computed using wave force response amplitude operators (ROA).Implementations of the ship model, Isherwood's wave model, wave force ROAs and the JONSWAP spectrum can be found in the Marine Systems Simulator toolbox by Fossen and Perez 4.The dataset is split into a routine operation set (96 hours) and into an Out-Of-Distribution (OOD) set (29 hours). The routine operation set is split into train-validation-test with a 60-10-30 split, while the OOD set is used solely for testing.The dataset is used for the evaluation of nonlinear system identification methods for multi-step predictions. The following inputs and outputs are considered for the identification problem. Inputs are the shaft speeds of both propellers, their azimut angles, wind angle of attack, and wind speed. Measured states or outputs are surge velocity, sway velocity and roll rate, as well as yaw angle and roll angle.Please see the README.txt file for details regarding the file structure of this dataset and a description of the variables in the .tab files.This research is funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy - EXC 2075 - 390740016. We acknowledge the support by the Stuttgart Center for Simulation Science (SimTech).1 T. Perez, A. Ross, and T. I. Fossen, “A 4-DOF SIMULINK model of acoastal patrol vessel for manoeuvring in waves,” in IFAC MCMC, 2006.2 R. M. Isherwood, “Wind resistance of merchant ships,” The RoyalInstitution of Naval Architects, 1972.3 K. Hasselmann, T. Barnett, E. Bouws, H. Carlson, D. Cartwright, K. Enke,J. Ewing, H. Gienapp, D. Hasselmann, P. Kruseman, A. Meerburg,P. Muller, D. Olbers, K. Richter, W. Sell, and H. Walden, “Measurementsof wind-wave growth and swell decay during the joint north sea waveproject (JONSWAP),” Deut. Hydrogr. Z., vol. 8, pp. 1-95, 01 1973.4 T. I. Fossen and T. Perez, “Marine Systems Simulator (MSS),” https://github.com/cybergalactic/MSS, 2004, last accessed: 2022-06-14.

Abstract

Recurrent neural networks are capable of learning the dynamics of an unknown nonlinear system purely from input-output measurements. However, the resulting models do not provide any stability guarantees on the input-output mapping. In this work, we represent a recurrent neural network as a linear time-invariant system with nonlinear disturbances. By introducing constraints on the parameters, we can guarantee finite gain stability and incremental finite gain stability. We apply this identification method to learn the motion of a four-degrees-of-freedom ship that is moving in open water and compare it against other purely learning-based approaches with unconstrained parameters. Our analysis shows that the constrained recurrent neural network has a lower prediction accuracy on the test set, but it achieves comparable results on an out-of-distribution set and respects stability conditions.

Abstract

The modern world is highly influenced by user-contributed content such as user comments, which is one of the most popular forms of communication on social media. They help build a connection between content creators and content consumers, as well as a connection between users of a social platform, which makes them highly relevant for community interaction. However, researchers have not treated users' comment reading strategies in much detail. The existing accounts are limited to address solely an explicit comment reading approach, which is only able to track an active interaction, e.g. analyze attention by counting a number of clicks. This technique can not fully estimate the drawn attention, as 73% of people do not actively interact with comments. Eye-tracking plays a vital role in solving the task of analysing implicit attention, as it allows to estimate such crucial characteristics, as comment features, which drew the most attention, the amount of attention given or the order of seeing specific comments. The current research uses eye-tracking based attention analysis for investigating the phenomena of users' reading behaviour on the real YouTube interface. The present master thesis concentrates on analysing users' attention mechanisms and reading behaviour and finding a correlation between the comment features such as length, language, sparseness, comment position, number of likes, presence of replies, presence of video creator's like and authorised user label, which cannot be exhibited from a pure textual point of view. This work shows that number of likes and presence of answers contribute the most to the attention drawn by comments. The analysis revealed that the category of a video deeply influences the emotion and length of the popular comments: if people are looking for useful information, which applies to educational videos, they tend to pay attention to neutral and long comments, whereas while looking for entertainment, people most probably will notice short and positive comments. The two important findings from the gender analysis are that women tend to read longer comments, but skip a higher percentage of comments than men. It is hoped that this research will contribute to a deeper understanding of the features that draw the most attention, which can be exploited in content generation strategies and developing new ranking algorithms.

Abstract

Graph Convolutional Networks (GCNs) are typically studied through the lens of Euclidean geometry. Non-Euclidean Riemannian manifolds provide specific inductive biases for embedding hierarchical or spherical data, but cannot align well with data of mixed topologies. We consider a larger class of semi-Riemannian manifolds with indefinite metric that generalize hyperboloid and sphere as well as their submanifolds. We develop new geodesic tools that allow for extending neural network operations into geodesically disconnected semi-Riemannian manifolds. As a consequence, we derive a principled Semi-Riemannian GCN that first models data in semi-Riemannian manifolds of constant nonzero curvature in the context of graph neural networks. Our method provides a geometric inductive bias that is sufficiently flexible to model mixed heterogeneous topologies like hierarchical graphs with cycles. Empirical results demonstrate that our method outperforms Riemannian counterparts when embedding graphs of complex topologies.

Abstract

Despite the large surges in technology investment and the dominance of smart systems, some emergency events remain unsolved or solved with a massive trade-off between performance and portability. The sudden fire outbreak that may occur due to several reasons is one of the problems that is yet to be solved, a natural disaster that could cause massive ecological and economic losses. Therefore, it is crucial to develop precise, fast, and commercially available solutions. Several image classification approaches were effectively designed, However, most of these models contain many layers that often lead to a massive model size such as AlexNet and GoogleNet. In this paper, we propose a model inspired by FireNet that balances and softens the tradeoff between performance and model size. In this work, smoke detection is augmented to FireNet, the model is trained and tested on various datasets and has an accuracy of 83% which is quite promising as compared to other CNN models that are deployed on an embedded device such as raspberry pi.

Abstract

Person Re-Identification is a challenging task in the field of computer vision, it has gained a huge space due to its importance in our life. Person Re-ID is mainly utilized in security applications, it is used everywhere like airports, metro stations, banks, etc. Due to its crucial importance many approaches were proposed to perform such task, the explosion of deep learning and its ability to outperform most of the state of-the-art approaches motivated researchers to use it in person re-identification, moreover researchers introduced benchmark person re-id datasets to train these models, which were able to provide remarkable results and provide a solid benchmark for comparison. The only limitation to all these models and datasets is that they were built assuming no pedestrian clothes variations. In our work we aim to address the problem of Long Term Person Re-Identification where the clothes variation exists. We start by experimenting and comparing the most recent state-of-the-art deep learning models that were developed for that task such as OSNet and PCB. After that we introduce our proposal to replace the backbone network of PCB with OSNet to act as a feature extractor. All of these models were trained and tested using NKUP dataset and Market-1501. The benchmark dataset Market-1501 was used to compare our model’s performance with several standard models. Our model was able to outperform the" PCB" model on Market-1501 and improve the accuracy from 87.71% to 91.3%. On the NKUP dataset our model improved the accuracy of" PCB" model from 16.1% to 17.6%.

Abstract

Leveraging deep learning models for Anomaly Detection (AD) has seen widespread use in recent years due to superior performances over traditional methods. Recent deep methods for anomalies in images learn better features of normality in an end-to-end self-supervised setting. These methods train a model to discriminate between different transformations applied to visual data and then use the output to compute an anomaly score. We use this approach for AD in text, by introducing a novel pretext task on text sequences. We learn our DATE model end-to-end, enforcing two independent and complementary self-supervision signals, one at the token-level and one at the sequence-level. Under this new task formulation, we show strong quantitative and qualitative results on the 20Newsgroups and AG News datasets. In the semi-supervised setting, we outperform state-of-the-art results by +13.5\% and +6.9\%, respectively (AUROC). In the unsupervised configuration, DATE surpasses all other methods even when 10\% of its training data is contaminated with outliers (compared with 0\% for the others).

Abstract

Physical motion models offer interpretable predictions for the motion of vehicles. However, some model parameters, such as those related to aero- and hydrodynamics, are expensive to measure and are often only roughly approximated reducing prediction accuracy. Recurrent neural networks achieve high prediction accuracy at low cost, as they can use cheap measurements collected during routine operation of the vehicle, but their results are hard to interpret. To precisely predict vehicle states without expensive measurements of physical parameters, we propose a hybrid approach combining deep learning and physical motion models including a novel two-phase training procedure. We achieve interpretability by restricting the output range of the deep neural network as part of the hybrid model, which limits the uncertainty introduced by the neural network to a known quantity. We have evaluated our approach for the use case of ship and quadcopter motion. The results show that our hybrid model can improve model interpretability with no decrease in accuracy compared to existing deep learning approaches.

Abstract

Argumentation Frameworks are an approach of formalizing arguments and their interrelations in a graph structure. They can be used to draw conclusions from this modelling of knowledge. Since argumentation is an important part of human reasoning, these graph structures can be considered easily interpretable, what makes this technology an interesting explainable artificial intelligence method. Although this is not their main purpose, Quantitative Argumentation Frameworks can be used to solve classification problems by following a new approach. This approach is based on constructing them out of multilayer perceptrons (MLP), based on the work of Potyka. In this thesis we were motivated to construct Quantitative Argumentation Frameworks out of sparse MLPs. A swarm intelligence algorithm, namely Particle Swarm Optimization (PSO), was developed to search for sparse MLP models with specific characteristics that relate to performance and topology of the graphical structures. Models were implemented, tested, and evaluated on three different datasets. The implementation includes preprocessing of the datasets, parameter learning of MLPs based on backpropagation, and structure learning of the MLP graphical structures. The evaluation involves constructing fully connected MLPs and decision trees for comparison purposes. The resulting models achieved high performance and low complexity in their structure. The PSO algorithm also proved its efficiency in solving the structure learning.

Abstract

Multistep prediction is the prediction of states based on initial states and a series of control inputs. This paper focuses on developing transformer models for multistep prediction of vehicle states and testing different modifications of the transformer architecture using the example of the prediction of a ship simulation. Research in NLP promises advantages with regard to training time and prediction accuracy for the transformer architecture compared to a state-of-the-art LSTM model. The author also investigates whether positional encodings are useful in this scenario and if a transformer model can learn the order of the inputs without positional encodings.

Abstract

The extraction and segmentation of references from scientific articles is a core task of modern digital libraries. Once references are extracted and segmented, the bibliographic information can be made publicly available and linked, enabling efficient literature study. However, references often vary in their structure and content. This makes the extraction and segmentation of references a challenging but valuable task. The purpose of this thesis is to investigate whether Bidirectional Encoder Representations from Transformers (BERT) is suitable for the extraction and segmentation of bibliographic references. Therefore, we follow a deep learning approach for the extraction and segmentation of references from PDF documents. We use a neural network architecture based on BERT, a deep language representation model that has significantly increased performance on many natural language processing tasks. Over the BERT output, we put a linear-chain Conditional Random Field. We experiment with different BERT models and input formats and also examine two approaches for reference extraction and segmentation. The experiments are evaluated on a challenging dataset that contains both English and German social science publications with highly varying references. Our results show that the best performing BERT models were pre-trained on similar data to the data that we used for the fine-tuning of the BERT models on the task of reference extraction and reference segmentation. Moreover, our findings show that long, context-based input sequences yield the best results. The extraction model identifies and extracts references with an average F1-score of 81.9%. References are segmented with an average F1-score of 93.6%. We show that our models compare well to one other previously published work. Our conclusion is that BERT is a suitable choice for reference extraction and reference segmentation.

Abstract

Today, small satellites are an increasingly important way for researchers to launch experiments and payloads into Earth orbit. On the one hand, these satellite systems have a growing need for data transmission to a ground station; on the other hand, various constraints apply to the design and operation of radio communication systems in space. Today's radio systems usually have static transceiver configurations and the transmission rates used correspond to the worst-case consideration for the particular case. Recently, adaptive approaches for the operation of radio links between satellite and ground station have been described, in which the transceiver configuration, controlled by a learning algorithm, is changed during operation and adapted to the respective external influences. The basis for the algorithms used is the reinforcement learning model. The goal of this work was the further development of an algorithm developed for satellite communication and the testing of this algorithm within a simulation environment in order to evaluate the effects of the changes. The modified algorithm was run in a software simulation environment that included the digital signal processing of the transmitter and receiver and a model for the changing transmission conditions during a satellite overflight. The modification was compared with the original algorithm with respect to the achievable data transmission. Among other things, the number of neural networks used for learning was varied. Furthermore, different hyperparameters were varied and the effects on the data transmission were examined. An adjustment of the hyperparameters led to the transmission of 57.8% more data than in the baseline implementation of the algorithm. Both, the change in architecture and the reduction in the number of neural networks executed in parallel, led to slight performance losses of 4.0% and 3.3%, respectively. The results of the software simulations performed here, however, cannot be directly transferred to hardware simulation environments or actual communication links. The implementation of the different algorithm variants allows to test these variants with hardware transmission links in the future and to test such a system on a small satellite later on.

Abstract

Playing computer games is one of the most common and enjoyable activities in the modern lifestyle. However, people with motor impairments who cannot use mouse and keyboard are not able to interact with computing devices as required by the game design. In this regard, novel interaction techniques can enable hands-free control using gaze and voice as input modalities to assist people with motor and speech impairments. This is the first evaluation of a video game control method consisting of a combination of eye tracking and non-verbal voice commands (e.g. humming) applied in a 2D jump-and-run game environment involving essential spatio-temporal interactions. To assess interaction feasibility, the evaluation study consisted of both qualitative and quantitative measures. In addition, the feasibility was validated with a target user group of people with motor impairments. Overall, the results indicate a lower but competitive performance while increasing the fun factor.

Abstract

Misinformation in online media is a broad term to design deceitful content, such as disinformation (i.e. fake news), rumors, manipulated content, authentic material used in the wrong context 6. While misinformation in general may not be driven all the time by the intent to deceit, disinformation has indeed such aim 2.Under any form, however, misinformation undeniably poses a threat, as this content can maliciously manipulate peoples beliefs and their decisions, carrying thus a social impact. For example, misinformation about the refugee crisis affected how citizens view refugees and their attitudes towards national and European Union politics 1. Countering the

Abstract

Social media have created communication channels between citizens and policymakers but are also susceptible to rampant misinformation. This new context demands new social media policies that can aid policymakers in making evidence-based decisions for combating misinformation online. This paper reports on data collected from policymakers in Austria, Greece, and Sweden, using focus groups and in-depth interviews. Analyses provide insights into challenges and identify four important themes for supporting policy-making for combating misinformation: a) creating a trusted network of experts and collaborators, b) facilitating the validation of online information, c) providing access to visualisations of data at different levels of granularity, and d) increasing the transparency and explainability of flagged misinformative content. These recommendations have implications for rethinking how revised social media policies can contribute to evidence-based decision-making.

Abstract

Composition models of distributional semantics are used to construct phrase representations from the representations of their words. Composition models are typically situated on two ends of a spectrum. They either have a small number of parameters but compose all phrases in the same way, or they perform word-specific compositions at the cost of a far larger number of parameters. In this paper we propose transformation weighting (TransWeight), a composition model that consistently outperforms existing models on nominal compounds, adjective-noun phrases, and adverb-adjective phrases in English, German, and Dutch. TransWeight drastically reduces the number of parameters needed compared with the best model in the literature by composing similar words in the same way.

Abstract

Prepostitional phrase (PP) attachment is a well known challenge to parsing. In this paper, we combine the insights of different works, namely: (1) treating PP attachment as a classification task with an arbitrary number of attachment candidates; (2) using auxiliary distributions to augment the data beyond the hand-annotated training set; (3) using topological fields to get information about the distribution of PP attachment throughout clauses and (4) using state-of-the-art techniques such as word embeddings and neural networks. We show that jointly using these techniques leads to substantial improvements. We also conduct a qualitative analysis to gauge where the ceiling of the task is in a realistic setup.

Abstract

In this paper, we compare various options for reifying RDF triples. We are motivated by the goal of representing Wikidata as RDF, which would allow legacy Semantic Web languages, techniques and tools – for example, SPARQL engines – to be used for Wikidata. However, Wikidata annotates statements with qualifiers and references, which require some notion of reification to model in RDF. We thus investigate four such options: (1) standard reification, (2) n-ary relations, (3) singleton properties, and (4) named graphs. Taking a recent dump of Wikidata, we generate the four RDF datasets pertaining to each model and discuss high-level aspects relating to data sizes, etc. To empirically compare the effect of the different models on query times, we collect a set of benchmark queries with four model-specific versions of each query. We present the results of running these queries against five popular SPARQL implementations: 4store, BlazeGraph, GraphDB, Jena TDB and Virtuoso.

Abstract

This paper presents a language-independent annotation scheme for the semantic relations that link the constituents of noun-noun compounds, such as Schneemann `snow man' or Milchmann `milk man'. The annotation scheme is hybrid in the sense that it assigns each compound a two-place label consisting of a semantic property and a prepositional paraphrase. The resulting inventory combines the insights of previous annotation schemes that rely exclusively on either semantic properties or prepositions, thus avoiding the known weaknesses that result from using only one of the two label types. The proposed annotation scheme has been used to annotate a set of 5112 German noun-noun compounds. A release of the dataset is currently being prepared and will be made available via the CLARIN Center Tübingen. In addition to the presentation of the hybrid annotation scheme, the paper also reports on an inter-annotator agreement study that has resulted in a substantial agreement among annotators.

Abstract

We investigate whether and in what conditions pronominal anaphora could be acquired by intelligent agents as a means to express recently mentioned entities. The use of pronouns is conditioned by the existence of a memory recording the object previously in focus. The approach follows an evolutionary paradigm of language acquisition. Experiments show that pronouns can be easily included in a vocabulary of a community of 10 agents dialoguing on a static scene and that, generally, they enhance the communication success.