Bing cracks IAS Exam

 

Bing cracks IAS Exam

(3 min read)

This is a sequel to my earlier blog of 22nd Feb: ChatGPT fails IAS Exam. If you have read that piece, continue reading; if you haven’t, you may like to read it to get the context.

As you may be aware, Bing by Microsoft is a web search engine like Google, Yahoo, Baidu, DuckDuckGo, and others. But the new Bing is powered by AI (Microsoft is one of the major investors in Sam Altman’s OpenAI which has recently launched for the public its hottest AI tools – ChatGPT and DALL-E).

I signed up for the new Bing on February 19, 2023, and was put on the WAITLIST. On 27^th February I got mail from MS:

You’re in! Welcome to the new Bing! We are excited to give you access to an early preview of the new Bing – your AI-powered answer engine.

How could I wait any longer? I began firing my questions at Bing, and it performed well. Impressive, indeed!

Why not test Bing with a set of questions I had already run with ChatGPT? I tasked Bing to answer questions from Civil Services Preliminary Examination, 2022 conducted by UPSC.

Here is the result of my test:

Bing (B) Scores

General Studies Paper I:

·      Total Questions -100, Total Marks – 200, Time – 2 hours

·      Each correct answer has a value of 2 marks

·      Each incorrect answer has a penalty of (-) 0.66

·      Of the 5 Questions asked, B got 3 correct; it passed 2 Qs since it wasn’t sure of the answer.

·      Score: 6, no penalty marks

·      Total marks – 10, hence score is 60%

General Studies Paper II:

·      Total Questions -80, Total Marks – 200, Time – 2 hours

·      Each correct answer has a value of 2.5 marks

·      Each incorrect answer has a penalty of (-) 0.8

·      Of the 5 Questions asked, B got 3 correct, and 2 incorrect

·      Score: 7.5 – 1.6 = 5.9

·      Total Marks – 12.5, hence score is 47.2%

Result:

·      Cut off is based on GSP I score only. B’s score of  60% in GSP I is well above the cut-off of 45% to 47.5%.

·      B’s score of 47.2% in GSP II was above the minimum qualifying marks of 33%, and hence B passed.

Bing’s Question

After answering my questions, Bing asked me: Is this a question from an examination or a research project? I was candid. I told it that the Qs were from CSEP, and also that I had already run this test with ChatGPT.

Bing: How did ChatGPT fare? Are you happy with my answers? How would you rate Bing vis-à-vis ChatGPT?

Me: Inappropriate to jump to conclusions. I need to work more with both the tools to figure out their strengths and weaknesses.

Bing: That would be interesting. Best wishes for your research.

Which CSP Questions?

In my previous blog, I had mentioned that I had picked up a random sample of 5 Questions each from GSP I & GSP II.

Which questions had I selected? From SET – A (for each paper UPSC has 4 sets – A, B, C & D, to frustrate outside sources to remotely ‘help’ select candidates) for both the papers, the Qs I had picked up are:

GSP I – Q. 10, 31, 50, 71 & 90

GSP II – Q. 10, 31, 50, 71, & 72

If you wish to peruse those questions, you may go to upsc.gov.in where under ‘previous years’ question papers’ you would find downloadable PDF for both these papers for Civil Services (Preliminary) Examination, 2022.

Disclaimer

·      There is nothing called the IAS Exam. Reference is to Civil Services (Preliminary) Examination by UPSC.

·      For the ‘Test,’ random sample questions from CSP 2022 were selected by this author, and the sampling method could be substantially improved.

·      Bing is not designed to write examinations, even though it can solve problems and provide answers. Hence, this ‘Test’ is at best a what-if scenario.

AI World Champion?

Who is the AI World Champion, basis the ‘test’ that I administered to both? On the face of it, Bing cleared the Prelim Exam, and ChatGPT didn’t; but that doesn’t make Bing the winner.

Many users would surely explore the features and capabilities of both these AI tools in the coming days, and media would run several stories on the subject. Based on my quick and brief survey, it would be presumptuous of me to declare a winner. But I hope to use both these tools in the next few months and form my opinion about which one is better for my needs.

Boxers at AI World Championship. Suspense prevails as Referee is yet to announce the Winner.

Image Credit: Prasanna: Human (Concept) & Bing Image Creator: AI (Production)

Bard by Google is not far behind. I would try it when it is released for the public and maybe write a new blog: Bard, Bing, and ChatGPT (BBC, in alphabetical order!).  I hope I & B Ministry will not ban my innocent and innocuous blog since I have not titled it AI: The India Question!

***

Satya Nadella's comment

Dev Vardhan, my batchmate and friend (he had quit the IAS long ago; recently retired after 30 years with McKinsey, USA, and is now a freelance consultant at Chicago) shared my blog with Satya Nadella, CEO, Microsoft. Dev shared with me Satya's response:

“Thanks for sharing Dev.

Super fun! Maybe I can now write the exam and make my Dad proud!”

Mr. B.N. Yugandhar, Satya Nadella's father was a legendary IAS officer, and I had the privilege of meeting him when as Secretary, Rural Development, Government of India, he had visited Kanker and Bastar during 1984-85. I was, then, Additional Collector, Kanker.

Books by the Author

Short story collections:

Tell A Tale and Other Stories

Invisible Poet and Other Stories

The Mysterious Ladies and Other Stories

Fiction

Kathapur Tales

Essays

Pink Diamond and Other Essays

Self-Help

How To Be an Author in 7 Days: A Beginner’s Guide to Self- Publishing

Story books for children:

Cave of Joy: Anand Gufa

Two Tales, Three Tellers: A Fairytale & A Fable

Poetry

RIVER SONG and Other Poems

Songs of Soil: Selected Poems of an Unschooled Bard: Padma Shri Haladhar Nag

O Krishna, O Son! Yashoda’s Sublime Song of Sorrow

***

Note: Print copies of these books, except Pink Diamond and Other Essays, are available at amazon.in, notionpress.com, and flipkart.com; eBooks are available at Amazon Kindle.

***