SharpCap Knowledge Base
Posted: Tue Nov 18, 2025 2:44 pm
Hi Robin,
Retrieval Augmented Generation (RAG) is a technique for rapidly querying a very large custom knowledge base and then using ChatGPT to summarize the findings in a friendly conversational manner.
For my knowledge base, I used the NASA Astrophysics database of thousands of published scientific papers. In SharpCap’s case, you would store all of your acquired knowledge and how it relates to your product.
I started by going to the NASA ADS website and asked it to return the top-100 abstracts having to do with “Cepheid Variable Stars” since that is my area of interest. Then I exported those abstracts into what’s called a Vector Database. I set up an account with a company called Qdrant. They offer a FREE 1GB cluster. This enables rapid retrieval of abstracts given a query like “Why observe Cepheids in infrared?” BTW, trying to use a conventional database is a fool’s errand.
In my application I accept a user query, then retrieve the top-3 ranked hits from Qdrant. Then I input those three abstracts to a call to ChatGPT to summarize it given the original query. Here is the result. The response is perfectly dry and scientific. Something to explore is to give it a personality (i.e. “AI Robin”):
-----------------------------------------------------------
Answer: Observing Cepheids in the infrared, particularly the near-infrared (NIR), offers several advantages:
1. **Tighter Period-Luminosity Relation**: The Cepheid period-luminosity relation is tighter at NIR wavelengths. This provides more precise distance measurements, which are crucial for the cosmic distance ladder. This information is supported by 2025AAS...24545506C and 2024IAUGA..32P1871C, which highlight the improved relation at longer wavelengths.
2. **Reduced Impact of Interstellar Extinction**: Interstellar extinction, which can obscure and redden light from stars, has less effect in the NIR band. This allows for clearer observations of Cepheids, thus enhancing the accuracy of inferred distances (2025AAS...24545506C and 2024IAUGA..32P1871C).
3. **Compatibility with Ground-Based Observatories**: NIR data can be acquired using ground-based telescopes, making it relatively accessible and cost-effective for studying Cepheids. This is mentioned in 2025AAS...24545506C and 2024IAUGA..32P1871C.
The detailed analyses and observations of Cepheids in these abstracts, including those performed with JWST which demonstrate enhanced source separation from line-of-sight companions, further emphasize the improved quality and reliability of NIR observations (2022ApJ...940L..17Y).
Based on the provided abstracts, these are the primary reasons why observing Cepheids in the infrared is advantageous.
-----------------------------------------------------------
Since this is a custom implementation I can integrate this into a Hybrid AI chat that couples RAG with functions to return answers to questions like "What is the pixel scale and CCD suitability of my kit" or "What is the minimum exposure of ...?".
Then I asked: “How did Henrietta Leavitt establish the foundation of Leavitt's Law which eventually led to the discovery that M31 is a separate galaxy and not a nebula in the Milky Way?"
-----------------------------------------------------------
Answer: Henrietta Leavitt's work laid the foundation of Leavitt's Law through her discovery of the relationship between the period and luminosity of variable stars (later identified as Cepheids) in the Small Magellanic Cloud, as published in 1912 (2025PASP..137d4001B). She meticulously measured light curves, periods, and magnitudes of these stars from photographic plates. This relationship, known today as the Leavitt Law, allowed for the measurement of extragalactic distances, transforming those variable stars into "standard candles" of cosmological distance measurement.
While none of the provided abstracts directly discuss how Leavitt's Law led to the realization that M31 (Andromeda) is a separate galaxy, the abstract 2025PASP..137d4001B indicates that Leavitt's Law was pivotal for accurate distance measurements, which later contributed to setting the scale of the Universe. This work paved the way for Edwin Hubble, who used Cepheid variables to measure distances to galaxies, ultimately establishing that M31 is indeed a separate galaxy beyond the Milky Way.
Based on the provided abstracts, I cannot fully answer if the specific process by which M31 was identified as a separate galaxy is discussed, as they do not include detailed events or studies about that specific discovery.
-----------------------------------------------------------
It's important to note that I've instructed ChatGPT to respond only to abstracts that I give it. This is called the "system prompt":
----------------------------------------------------------
You are a cautious astrophysics assistant.
You must answer ONLY using the paper abstracts provided in the current
conversation. Treat these abstracts as your entire universe of
knowledge for this question.
Rules:
1. Use ONLY information that appears in the provided abstracts.
- Do NOT use any outside knowledge, training data, or assumptions.
- You must ignore any astrophysics knowledge you may have learned
during training. Only the content of the provided abstracts is allowed.
- If a fact is not stated or clearly implied by an abstract, you
must not assert it.
2. If the abstracts do not contain enough information to answer the
question, say so explicitly. For example:
"Based on the provided abstracts, I cannot fully answer X because
they do not discuss Y."
3. When you use information from an abstract, mention its bibcode
when helpful (e.g., "2024IAUGA..32P1871C") so the user can trace
the source.
4. Do NOT invent bibcodes, paper titles, numerical values, or
references that are not present in the abstracts.
5. Every scientific claim in your answer must be directly supported by
a sentence or phrase in one of the provided abstracts.
6. If different abstracts appear to disagree, describe the difference
instead of silently choosing one side.
Your goal is to:
- Read the question.
- Carefully read the abstracts.
- Synthesize an answer that is fully grounded in those abstracts,
or honestly say that the abstracts are insufficient.
- Every scientific claim in your answer must be directly supported by a sentence
or phrase from one of the abstracts. If not, do not state it.
---------------------------------------------------------------
Brian
Retrieval Augmented Generation (RAG) is a technique for rapidly querying a very large custom knowledge base and then using ChatGPT to summarize the findings in a friendly conversational manner.
For my knowledge base, I used the NASA Astrophysics database of thousands of published scientific papers. In SharpCap’s case, you would store all of your acquired knowledge and how it relates to your product.
I started by going to the NASA ADS website and asked it to return the top-100 abstracts having to do with “Cepheid Variable Stars” since that is my area of interest. Then I exported those abstracts into what’s called a Vector Database. I set up an account with a company called Qdrant. They offer a FREE 1GB cluster. This enables rapid retrieval of abstracts given a query like “Why observe Cepheids in infrared?” BTW, trying to use a conventional database is a fool’s errand.
In my application I accept a user query, then retrieve the top-3 ranked hits from Qdrant. Then I input those three abstracts to a call to ChatGPT to summarize it given the original query. Here is the result. The response is perfectly dry and scientific. Something to explore is to give it a personality (i.e. “AI Robin”):
-----------------------------------------------------------
Answer: Observing Cepheids in the infrared, particularly the near-infrared (NIR), offers several advantages:
1. **Tighter Period-Luminosity Relation**: The Cepheid period-luminosity relation is tighter at NIR wavelengths. This provides more precise distance measurements, which are crucial for the cosmic distance ladder. This information is supported by 2025AAS...24545506C and 2024IAUGA..32P1871C, which highlight the improved relation at longer wavelengths.
2. **Reduced Impact of Interstellar Extinction**: Interstellar extinction, which can obscure and redden light from stars, has less effect in the NIR band. This allows for clearer observations of Cepheids, thus enhancing the accuracy of inferred distances (2025AAS...24545506C and 2024IAUGA..32P1871C).
3. **Compatibility with Ground-Based Observatories**: NIR data can be acquired using ground-based telescopes, making it relatively accessible and cost-effective for studying Cepheids. This is mentioned in 2025AAS...24545506C and 2024IAUGA..32P1871C.
The detailed analyses and observations of Cepheids in these abstracts, including those performed with JWST which demonstrate enhanced source separation from line-of-sight companions, further emphasize the improved quality and reliability of NIR observations (2022ApJ...940L..17Y).
Based on the provided abstracts, these are the primary reasons why observing Cepheids in the infrared is advantageous.
-----------------------------------------------------------
Since this is a custom implementation I can integrate this into a Hybrid AI chat that couples RAG with functions to return answers to questions like "What is the pixel scale and CCD suitability of my kit" or "What is the minimum exposure of ...?".
Then I asked: “How did Henrietta Leavitt establish the foundation of Leavitt's Law which eventually led to the discovery that M31 is a separate galaxy and not a nebula in the Milky Way?"
-----------------------------------------------------------
Answer: Henrietta Leavitt's work laid the foundation of Leavitt's Law through her discovery of the relationship between the period and luminosity of variable stars (later identified as Cepheids) in the Small Magellanic Cloud, as published in 1912 (2025PASP..137d4001B). She meticulously measured light curves, periods, and magnitudes of these stars from photographic plates. This relationship, known today as the Leavitt Law, allowed for the measurement of extragalactic distances, transforming those variable stars into "standard candles" of cosmological distance measurement.
While none of the provided abstracts directly discuss how Leavitt's Law led to the realization that M31 (Andromeda) is a separate galaxy, the abstract 2025PASP..137d4001B indicates that Leavitt's Law was pivotal for accurate distance measurements, which later contributed to setting the scale of the Universe. This work paved the way for Edwin Hubble, who used Cepheid variables to measure distances to galaxies, ultimately establishing that M31 is indeed a separate galaxy beyond the Milky Way.
Based on the provided abstracts, I cannot fully answer if the specific process by which M31 was identified as a separate galaxy is discussed, as they do not include detailed events or studies about that specific discovery.
-----------------------------------------------------------
It's important to note that I've instructed ChatGPT to respond only to abstracts that I give it. This is called the "system prompt":
----------------------------------------------------------
You are a cautious astrophysics assistant.
You must answer ONLY using the paper abstracts provided in the current
conversation. Treat these abstracts as your entire universe of
knowledge for this question.
Rules:
1. Use ONLY information that appears in the provided abstracts.
- Do NOT use any outside knowledge, training data, or assumptions.
- You must ignore any astrophysics knowledge you may have learned
during training. Only the content of the provided abstracts is allowed.
- If a fact is not stated or clearly implied by an abstract, you
must not assert it.
2. If the abstracts do not contain enough information to answer the
question, say so explicitly. For example:
"Based on the provided abstracts, I cannot fully answer X because
they do not discuss Y."
3. When you use information from an abstract, mention its bibcode
when helpful (e.g., "2024IAUGA..32P1871C") so the user can trace
the source.
4. Do NOT invent bibcodes, paper titles, numerical values, or
references that are not present in the abstracts.
5. Every scientific claim in your answer must be directly supported by
a sentence or phrase in one of the provided abstracts.
6. If different abstracts appear to disagree, describe the difference
instead of silently choosing one side.
Your goal is to:
- Read the question.
- Carefully read the abstracts.
- Synthesize an answer that is fully grounded in those abstracts,
or honestly say that the abstracts are insufficient.
- Every scientific claim in your answer must be directly supported by a sentence
or phrase from one of the abstracts. If not, do not state it.
---------------------------------------------------------------
Brian