Daniel Biś

Daniel Biś

Ph.D. Student

Florida State University

Biography

I am a second-year Ph.D. student in the Computer Science department at Florida State Universtiy, where I am lucky to be advised by Professor Xiuwen Liu. During my studies, I would like to answer fundamental questions related to enabling machines to become more adept at human language. My research area includes natural language processing, deep learning and intelligent systems in general. Currently, I am focused on self-supervised representation learning, analysis of representations learned by language models, word sense disambiguation, text summarization.

In the past, I completed an internship at Samsung R&D, where I focused on advancing Dialog Systems, and worked as a Data Scientist at Risk Managment Solutions, where I contributed to the development of HWind products.

In May of 2021, I will be joining Amazon Alexa AI team as an Applied Scientist Intern.

Outside of academia, swimming has been my lifelong passion. I was a Polish National Championship medalist, Poland’s National Swim Team Member, and competed at NCAA Division I athletics as a member of the varsity swim team at Florida State University.

Interests

  • self-supervised representation learning
  • analysis of representations learned by language models
  • word sense disambiguation
  • text summarization

Education

  • Ph.D. in Computer Science, 2019-Present

    Florida State University

  • B.A. in Computer Science, 2019

    Florida State University

Recent Publications

Quickly discover relevant content by filtering publications.

Too Much in Common: Shifting of Embeddings in Transformer Language Models and its Implications

The success of language models based on the Transformer architecture appears to be inconsistent with observed anisotropic properties of representations learned by such models. We resolve this by showing, contrary to previous studies, that the representations do not occupy a narrow cone, but rather drift in common directions.

Effects of Architecture and Training on Embedding Geometry and Feature Discriminability in BERT

In this paper we investigate how the BERT architecture and its pre-training protocol affect the geometry of its embeddings and the effectiveness of its features for classification tasks. As an auto-encoding model, during pre-training, it produces representationsthat are context dependent and at the same time must beable to “reconstruct” the original input sentences. The complex interactions of the two via transformers lead to interesting geometric properties of the embeddings and subsequently affectthe inherent discriminability of the resulting representations. Our experimental results illustrate that the BERT models do not produce “effective” contextualized representations for words.
PDF

Biomedical word sense disambiguation with bidirectional long short-term memory and attention-based neural networks

In this paper, we propose two deep-learning-based models for supervised WSD - a model based on bi-directional long short-term memory (BiLSTM) network, and an attention model based on self-attention architecture.
PDF

An Analysis on the Learning Rules of the Skip-Gram Model

In this work, we derive the learning rules for the skip-gram model and establish their close relationship to competitive learning. In addition, we provide the global optimal solution constraints for the skip-gram model and validate them by experimental results.
PDF

Layered Multistep Bidirectional Long Short-Term Memory Networks for Biomedical Word Sense Disambiguation

In this paper, we propose a novel deep neural network architecture for supervised medical word sense disambiguation.
PDF
See all publications

Awards

2nd Award, Poster Contest, Florida State University CS Expo

Florida State University Nov 2018

Nancy Casper Hillis and Mark Hillis Undergraduate Research Grant

Nancy Casper Hillis and Mark Hillis Endowed Fund May 2018

Florida-Eastern European Linkage Scholarship

Florida/Eastern Europe Linkage Institute Aug 2016 – May 2019

ACC Academic Honor Roll

Atlantic Coast Conference May 2016 – May 2019

Projects

MStream

C++ re-implementation of the paper Model-based Clustering of Short Text Streams by Yin et al, published at KDD 2018

Storm-Risk

Together with my colleague Grzegorz Kakareko, during a programing competition, we created Storm-Risk - online application designed to estimate and mitigate the house damage caused by hurricane winds.

Contact

  • 95 Chieftan Way, Tallahassee, FL 32304
  • Office #446, 4th floor