Background: Who is the Client and what is the product/service?

Context: Scaling too fast

At guild.xyz the team is full of super talented developers.

The traction and the iterations are very fast, and the product decisions are shaped mostly by the investors and bigger “clients” feedbacks and needs. This is totally OK for an early-stage startup, but guild.xyz grew soo fast, that the need for proper user metrics emerged soon.

(part of) The Solution: Setting up a product funnel

My research pointed always in one direction: the main challenge in the guild.xyz-s onboarding is to teach users new concepts (eg.: roles and requierements on guild page) and their relations to existing entities (eg.: already existing Discord roles).

Guild.xyz has two main User Persona:

• Community owners (2% of the traffic)
• They will set up and manage the Guild page
• They need to understand the whole process
• They will communicate about it to the community members
• They want to make community management easier but in the same time they fear losing members who don’t understand / like the new access management flow
• Potential / existing community members (98% of the traffic)
• They see the link for the Guild page and enter to the community after a verification process
• If they don't trust the platform or don't understand something, they churn off or ask for help to the Community owner

I wanted to focus more on the latter. They are our “real-users”, and mediators to the members. This is why the understanding “Guild Creation Flow” was in higher priority than the “Join Guild Flow”, however, we gained lots of actionable insight, (than conversion improvement) from the latter also.

Steps:

1. Research and explore the flow.

Having real experience with the service is important, even if you think, you understand the whole process. During my criminal psychology studies, once a judge was invited as lecturer. He said, that before the trial of a murder case, he often goes for a walk to the crime scene, because the first hand experience always adds insights. The same is true for digital products.

2. Draft out the measurement points and discuss with stakeholders

After assessing needs with stakeholder interviews I made a draft funnel. The main goal was to create a medium for conversation. Thinking together is always easier visually.

3. Consult with the devs about custom event possibilities.

The tool we used, was more like an infrastructure-measurement system, instead of product analytics.

So I had to adapt to this and facilitate the solution for creating custom events.

4. Review and test the events, whether they really work as intended.

Of course, it doesn't. :) Its tricky, because sometimes the metrics looks like they working, but some of the metrics point makes no sense. But with the collaboration of a dev collage, we were able to find and fix bugs.

5. Build a workflow for curating custom events.

Sounds obvious, but it isn’t. This product changes fast. If we have an event that is outdated, we can't just delete it, because it is possible that we want to use it in the future for comparisons.

A possible solution for this is the very accurate tracking of the custom events and their timelines. In our case the medium for this was a gist table, curated by the dev who creates new events:

6. Discover some new insights.

After setting a metric system, and waiting for the first significant sample size after you weed out every bug, is full of excitement. Similar to waiting for the first conversion rates after a campaign you designed, but more intense.

We found lots of thing, but the main insights I want to show here was:

- Join flows weak points are the third-party services. Both Community owners and Community members need to connect their Ethereum wallet and their Discord.

7. Write an in-house Notion guide about the querryes.

My goal was more than deliver useful insights. I wanted to

Results

Shortly after the launch of bug fixes and features ( eg.: one-click Discord Join) and the new Onboarding flow, the success ratio was 66% higher on the Guild Creation Flow.

After waiting for a bigger sample size the improvement….improved to 18-20%.

The Join Guild Flow liked the changes also. Before:

After:

1. Van egy kis churn a wallet connectnél. Nekem megérzésre nagyobb mint amit a bugok magyaráznának.

Le tudjuk e valahogy szűrni, vagy legalább educated guessünk lehet-e arról, hogy a 20% dropoffnak milyen okai vannak?

2. A második churn már fájdalmasabb.

Azonban

@r4Z0.

rögtön rávágta hogy ennek az egyik oka az lehet, hogy a userek olyan guild pageket nézegetnek ahova nincs bejárásuk. Ez egy jó hipotézis de szuper lenne utána menni hogy milyen okok lehetnek még, nekem az a megérzésem h niincs enniy exploráró user. Ide tartozik még, hogy le lehetne szűrni erre a funnelre azokat, akik: - egy adott guild Join felhívására kitett linkről jönnek. (pl. twiitter felhívás, guild honlap felhívás ..stb) - A guildnek van free entry roleja. Így tuti olyan userek adatait látnánk a funnelen, akiknek volt

szándékuk és lehetőségük

belépni. Ergo: aki itt dropoffol az valami olyan okkal teszi, amin mi valahogy (IT / UX ) tudunk segíteni . Tehát a kérdés: Hogy lehet traffic sourcre-ra szűrni datadogon?

(Ha megtaláltam leírom ide) + Az is érédekel hogy milyen guildeket lenne érdemes így monitorozni.

(detto)

@April 1, 2022 4:22 PM (GMT)

k0baval atbeszeltuk a join guild funnelt, kb ezekre jutottunk:

1. dropoff 21%: nem connectal walletot a user, hipotezis: nincs epp olyan szituban hogy komfortos legyen a walletjaval operalni (eg. mobilon nem szokott - itt kulon sokkal nagyobb dropoffot lattunk, 3rd party wifin van, double csekkolja hogy legit-e a guild stb) - sztem ez ok, egyetlen action point amit el tudok kepzelni hogy twitteren/blogpostban edukaljuk az embereket a wallet hasznalatrol
2. dropoff 65%: replayeket visszanezve a fo ok valoszinuleg hogy nincs accesse a usernek. curious, korulnez, talan remeli hogy bejut valahova, de lepattan. amit csinalhatunk: datadogban mappelni hogy wallet connect utan a join button tenyleg join-e vagy no access, ezeket eventekkent hozzaadni a funnelhez
3. dropoff 30%: ez talan a legerdekesebb, itt a discord connecten buknak el az emberek. a replaybol nem latjuk hogy a dc oldalan benaznak-e vagy szimplan rajonnek hogy nem akarjak az on es off chain identityt osszekotni (ami valid erv), ezt lehetne meg tovabb vizsgalni