Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
prediction-market-agent-tooling
Advanced tools
Tools to benchmark, deploy and monitor prediction market agents.
Tooling for benchmarking, deploying and monitoring agents for prediction market applications.
Install the project dependencies with poetry
, using Python >=3.10:
python3.10 -m pip install poetry
python3.10 -m poetry install
python3.10 -m poetry shell
Create a .env
file in the root of the repo with the following variables:
Deploying and monitoring agents using GCP requires that you set up the gcloud CLI (see here for installation instructions, and use gcloud auth login
to authorize.)
MANIFOLD_API_KEY=...
BET_FROM_PRIVATE_KEY=...
OPENAI_API_KEY=...
Create a benchmarkable agent by subclassing the AbstractBenchmarkedAgent
base class, and plug in your agent's research and prediction functions into the predict
method.
Use the Benchmarker
class to compare your agent's predictions vs. the 'wisdom of the crowd' on a set of markets from your chosen prediction market platform.
For example:
import prediction_market_agent_tooling.benchmark.benchmark as bm
from prediction_market_agent_tooling.benchmark.agents import RandomAgent
from prediction_market_agent_tooling.markets.markets import MarketType, get_binary_markets
benchmarker = bm.Benchmarker(
markets=get_binary_markets(limit=10, market_type=MarketType.MANIFOLD),
agents=[RandomAgent(agent_name="a_random_agent")],
)
benchmarker.run_agents()
md = benchmarker.generate_markdown_report()
This produces a markdown report that you can use for comparing agents side-by-side, like:
Deprecated: We suggest using your own infrastructure to deploy, but you may still find this useful.
Create a deployable agent by subclassing the DeployableTraderAgent
base class, and implementing the answer_binary_market
method.
For example, deploy an agent that randomly picks an outcome:
import random
from prediction_market_agent_tooling.deploy.agent import DeployableTraderAgent
from prediction_market_agent_tooling.markets.agent_market import AgentMarket
class DeployableCoinFlipAgent(DeployableTraderAgent):
def answer_binary_market(self, market: AgentMarket) -> bool | None:
return random.choice([True, False])
DeployableCoinFlipAgent().deploy_gcp(...)
Agents can control funds via a wallet primary key only, or optionally via a Safe as well. For deploying a Safe manually for a given agent, run the script below:
poetry run python scripts/create_safe_for_agent.py --from-private-key <YOUR_AGENT_PRIVATE_KEY> --salt-nonce 42
This will output the newly created Safe in the terminal, and it can then be copied over to the deployment part (e.g. Terraform).
Note that salt_nonce
can be passed so that the created safe is deterministically created for each agent, so that, if the same salt_nonce
is used, the script will not create a new Safe for the agent, instead it will output the previously existent Safe.
You can then specify this agent's Safe address with the SAFE_ADDRESS
environment variable.
Monitor the performance of the agents deployed to GCP, as well as meta-metrics of the prediction market platforms they are deployed to.
This runs as a streamlit app on a localhost server, executed with:
PYTHONPATH=. streamlit run examples/monitor/monitor.py
Which launches in the browser:
The following prediction market platforms are supported:
Platform | Benchmarking | Deployment | Monitoring |
---|---|---|---|
Manifold | ✅ | ✅ | ✅ |
AIOmen | ✅ | ✅ | ✅ |
Polymarket | ✅ | ❌ | ❌ |
We have built clean abstractions for taking actions on the different prediction market platforms (retrieving markets, buying and selling tokens, etc.). This is currently undocumented, but for now, inspecting the AgentMarket
class and its methods is your best bet.
For example:
from prediction_market_agent_tooling.config import APIKeys
from prediction_market_agent_tooling.markets.agent_market import SortBy
from prediction_market_agent_tooling.markets.omen.omen import OmenAgentMarket
# Place a bet on the market closing soonest
market = OmenAgentMarket.get_binary_markets(limit=1, sort_by=SortBy.CLOSING_SOONEST)[0]
market.place_bet(outcome=True, amount=market.get_bet_amount(0.1))
# View your positions
my_positions = OmenAgentMarket.get_positions(user_id=APIKeys().bet_from_address)
print(my_positions)
# Sell position (accounting for fees)
market.sell_tokens(outcome=True, amount=market.get_bet_amount(0.095))
This API can be built on top of to create your application. See here for an example.
See the Issues for ideas of things that need fixing or implementing. The team is also receptive to new issues and PRs.
We use mypy
for static type checking, and isort
, black
and autoflake
for linting. These all run as steps in CI.
FAQs
Tools to benchmark, deploy and monitor prediction market agents.
We found that prediction-market-agent-tooling demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.