Ian Waddington

It's tonight! Please join the PyData Cambridge community to welcome Mayank P. (Astra Zeneca) with his talk "Image Registration: How to align images" and Ehecatl Antonio del Rio Chanona (Imperial College London) with "Data-driven optimization for expensive problems: a chemical reactor case study". This event is free and in-person only. Please signup on Meetup (link below) and reach out if you have any questions or wish to contribute in any form. We are always looking for speakers, attendees, and funding (company sponsorships, personal donations, etc.). Please also follow us and share this event! - MeetUp (event signup): https://lnkd.in/eT3FRzt6 - LinkedIn: PyData Cambridge - Twitter: https://lnkd.in/eCr9BNT5 - Location (Map): https://lnkd.in/evutQgnY #llm #ai #datascience #dataengineering #cambridgeuk #sponsorship

15

Really happy to share that our work on “Learning Metrics that Maximise Power for Accelerated A/B-tests” will appear in the Applied Data Science track of #KDD2024. We address a common problem in industry: how can we minimise type-II errors in online controlled experiments? Can we simply learn a metric combination that directly minimises this quantity? This project started out as “let’s implement the ideas from this great paper by Eugene Kharitonov et al.”, but we ended up proposing some extensions to their seminal work that seem to be quite effective, and are now used across ShareChat. Proud to have this work out there, let us know what you think. Additionally, we were assigned an Area Chair that clearly cared: initial low-confidence reviews prompted them to invite more reviewers (we got 8!), and the AC themselves correctly grilled our rebuttals to point out arguments they deemed insufficient. It’s easy to be positive about peer review when your paper gets accepted, but it seems there are still some unsung heroes out there... Work with Aleksei U.. ACM SIGKDD & Annual KDD Conference

116

9 Comments

https://lnkd.in/epNxY4fb I've been thinking about the way that large language models seem to be able to generalise to unseen problems lately and I've been thinking that the spectral bias of neural networks is probably a key component of that. Between this and the double descent problem, it's becoming increasingly clear that massively overparameterized neural networks are biased towards simple and smooth solutions to a problem. Often, the simplest representation of a domain is also the 'correct' one. Consider a dataset that consists exclusively of uniformly sized black dots against a white background. The most compact encoding that contains all the information for this dataset is just the x and y co-ordinates of the dot. These values also happen to be the real generator of the dataset. It’d be reasonable to expect a beta VAE to converge on this representation based on spectral bias and it would be the most generic encoding. Is a similar thing happening in large language models? When presented with conversations and papers about scientific matters where the next word depends on what is true, perhaps the simplest policy is to understand the topic and to model reality first - language second.

3

That is impressive.

14

Recently, I got to experiment on developing a Text vectorisation service for a continuous inflow of Tickets text data. We used Celery and E5 model to test the performance and tried to optimise on throughput. I have penned down my learnings and the issues I faced during the experiment Text vectorisation service using Celery and E5 model

21

Following on from last week's article, this week I cover Activation Functions

1

🚀 Exciting News! 🚀 Our article, "Parallel Machine Scheduling with Re-entrant Jobs with Consideration of Set-up Times," has been published in The Eurasia Proceedings of Science, Technology, Engineering & Mathematics (EPSTEM) as part of IConTES 2024! 🎉 🔍 In this study, we developed an integer programming model to reduce set-up times and enhance capacity utilization in parallel machines with re-entrant jobs, while considering machine eligibility.The model was successfully tested at a panel manufacturing company in Türkiye, yielding impressive results! I would like to thank all my team members and esteemed advisors for their contributions! 🙌 📖 Access the article here: https://lnkd.in/dyrBmVsr #Research #Optimization #Scheduling #SetUpTimes 

46

6 Comments

I have published two datasets this week. First one is consolidate, select and reformatting of a bunch of top Thai instruct dataset Second is Slim Orca partially(once state of the art dataset) translate to Thai using gpt-4o-mini https://lnkd.in/gq3pG_Ru https://lnkd.in/gMSWi_MT

8

We're searching for four motivated PhD students that we will support with free access to Shaipup (https://buff.ly/4hJSUfl ) our dedicated API platform that drives impact from researchers' code-based insights. If you know any interested PhD students, please share this post with them. If you are a PhD student, please write to us and tell us why you want to showcase your great computer code insights; maybe you want to let community or business groups not only read about your work but also let them try it out. Perhaps you're looking for a job and want to let employers interact and run your computer code. Or maybe you are writing your first journal paper and want to generate a DOI to share it with reviewers and let readers access your live code to see how your analysis actually runs. Or perhaps you just want help to get your computer code better organised so you feel more in control of all the complexity in the data and analyses for your thesis. Whatever the reason, let me know; send me an email to dave.swain@terracipher.com and in less than 200 words, tell me why you want to be able to drive greater impact from your computer code.

12

How can we FORGET private information from large #MachineLearning models? Thrilled to share our latest preprint addressing the problem of #Machine #Unlearning from an information theoretic perspective! For what its worth, I think this is my favourite paper we've written so far Our proposed method: 🎓 Is principled and built upon information theory concepts 🚀 Is fast, and never needs to see the training data 📈 Is competitive with SOTA performance for zero-shot unlearning 👇 Link to paper and code in comments with Kyle Fogarty, Stefan Schöpf, Cengiz Oztireli and Alexandra Brintrup

49

3 Comments

I've just written a blog post once again - I find it quite interesting to show some 'agents' I've seen in PyPI. One uses a nice Python's feature to trick static analysers - just execute functions by writing their names in Unicode. Looks like your code editor couldn't load a font with local characters (if you use only English, you've probably never experienced how irritating it is, but it happens constantly on some websites) https://lnkd.in/d3S6wNGC

10

📚 Are you new to GenAI? 🎯 Are you looking for a good starting point? I would suggest Nir Diamant's repo Prompt Engineering 22 examples, basic to advanced Prompt structures and applications Detailed tutorials with implementation guidelines https://lnkd.in/eMiQSDbs

152

5 Comments

🔍 Google's Gemma Scope: Illuminating the Inner Workings of Large Language Models 🧠 Google researchers have developed a powerful new tool called Gemma Scope, designed to shed light on how each layer in Gemma 2 large language models responds to input tokens. 📊 Key insights: - Sparse autoencoders (SAEs) can transform embeddings into interpretable representations - Each index in the transformed embedding corresponds to a distinct concept - SAE weights indicate the strength of each concept in the input Gemma Scope enables: - Manual and automatic labeling of concepts in each layer - Steering the model by adjusting SAE outputs to generate concept-specific text This groundbreaking tool paves the way for answering critical questions about LLMs, such as how fine-tuning and chain-of-thought prompting influence a model's internal representations. 🤔 #GemmaScope #LargeLanugageModels #TransformerInterpretability #GoogleAI #MachineLearningResearch

1

IT'S HAPPENING! Whether you 'believe' in quantum computing's future advantage or think it's a load of nonsense conspired by Big Physics to sell more photons... We have FREE OPEN SOURCE tools to help you migrate! Know this: Regulation in cryptography is changing, so you need to begin your migration NOW! Thankfully, there are two FREE OPEN SOURCE tools, which I've been involved in. The first is CryptoMon - a passive network cryptography monitor that you can use to find cryptography moving over the wire: https://lnkd.in/eiPH6hk4 We also have the additions that we worked with Microsoft and GitHub to make to scan your code and find the cryptography you're using there: https://lnkd.in/e6e3gxF4 And now for the obligatory file links to the NEW NIST PQC STANDARDS: Module-Lattice-Based Key-Encapsulation Mechanism Standard - ML-KEM (formerly Kyber): https://lnkd.in/ekEb88Sk Module-Lattice-Based Digital Signature Standard - ML-DSA (formerly Dilithium):  https://lnkd.in/eKifmdQi Stateless Hash-Based Digital Signature Standard - SLH-DSA (formerly SPHINCS+): https://lnkd.in/edUp_U7j

34

2 Comments

👉Anthropic has succeeded in interpreting the internal workings of AI (LLM)! Anthropic has succeeded in interpreting the internal workings of the Claude 3 Sonnet AI model, addressing concerns about AI being a black box. This breakthrough marks the first concrete interpretation of a commercial large-scale model. By mapping neuron activation patterns into millions of human-interpretable concepts, they created a visual conceptual map. This allows for a better understanding of the model's predictions and behaviors, enhancing safety by identifying potential harmful actions in advance.

1

Contextual understanding between chunks without the extra storage cost from models like ColBERT? Late chunking is such a simple solution, and a greatly explained blog post!

7