{ "cells": [ { "cell_type": "markdown", "id": "83a62f94", "metadata": { "tags": [] }, "source": [ "# Most frequent builds\n", "\n", "See what builds are most commonly invoked by developers, e.g. `clean assemble`, `test` or `check`. You can [set up the URL and a token for your Develocity instance](https://github.com/gabrielfeo/develocity-api-kotlin/blob/main/README.md#setup) and run this notebook as-is for your own project.\n", "\n", "This is a simple example of something you can do with the API. It could bring insights, for example:\n", "\n", "- \"Our developers frequently `clean` together with `assemble`. We should ask them why, because they shouldn't have to. Just an old habit from Maven or are they working around a build issue we don't know about?\"\n", "- \"Some are doing `check` builds locally, which we set up to trigger our notably slow legacy tests. We should suggest they run `test` instead, leaving `check` for CI to run.\"\n", "\n", "This notebook will take you through using develocity-api-kotlin in Jupyter, but it won't get into what a notebook is and how to run it. If you're not familiar with Jupyter:\n", "\n", "- [Kotlin for data science overview](https://kotlinlang.org/docs/data-science-overview.html)\n", "- [Kotlin for Jupyter notebooks](https://github.com/cheptsov/kotlin-jupyter-demo/blob/master/index.ipynb)\n", "\n", "Note: GitHub preview won't render tables or graphs. I recommend previewing this in the [online Jupyter nbviewer](https://nbviewer.org/github/gabrielfeo/develocity-api-kotlin/blob/main/examples/example-notebooks/MostFrequentBuilds.ipynb)." ] }, { "attachments": {}, "cell_type": "markdown", "id": "79661abe-8aea-48c2-9938-e9452c0161d0", "metadata": {}, "source": [ "## Setup\n", "\n", "Add libraries to use, via line magics. `%use` is a [line magic](https://github.com/Kotlin/kotlin-jupyter#line-magics) of the Kotlin kernel that can do much more than adding the library. To illustrate, this setup can be replaced with a single line magic.\n", "\n", "```kotlin\n", "@file:DependsOn(\"com.gabrielfeo:develocity-api-kotlin:2024.3.0\")\n", "\n", "import com.gabrielfeo.develocity.api.*\n", "import com.gabrielfeo.develocity.api.model.*\n", "import com.gabrielfeo.develocity.api.extension.*\n", "```\n", "\n", "is the same as:\n", "\n", "```\n", "%use develocity-api-kotlin(version=2024.3.0)\n", "```" ] }, { "cell_type": "code", "execution_count": 1, "id": "97aad45e-fdb0-4ca3-8049-cfa7ce934bbb", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:24:43.176052Z", "start_time": "2024-04-01T20:24:41.907349Z" } }, "outputs": [], "source": [ "%useLatestDescriptors\n", "%use develocity-api-kotlin(version=2024.3.0)\n", "%use coroutines(v=1.7.1)\n", "\n", "val api = DevelocityApi.newInstance()" ] }, { "cell_type": "markdown", "id": "9d0556e7", "metadata": {}, "source": [ "## Fetch builds\n", "\n", "Use [getBuildsFlow][1] to fetch all builds for a [query][2] with `/api/builds`.\n", "\n", "By default, \"builds\" from the API are just an ID and an upload time, but we can request more info to come in the same \n", "response using the `models` parameter. \"Models\" are build details that would come from other endpoints. For example, \n", "requesting models=[gradleAttributes][3] brings data from `/api/builds/{id}/gradle-attributes` in the same response.\n", "\n", "[1]: https://gabrielfeo.github.io/develocity-api-kotlin/library/com.gabrielfeo.develocity.api.extension/get-builds-flow.html\n", "[2]: https://docs.gradle.com/enterprise/api-manual/#advanced_search_syntax \n", "[3]: https://gabrielfeo.github.io/develocity-api-kotlin/library/com.gabrielfeo.develocity.api\n", ".model/-build-model-name/gradle-attributes/index.html" ] }, { "cell_type": "code", "execution_count": 2, "id": "588a699f", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:24:45.458079Z", "start_time": "2024-04-01T20:24:43.318588Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "17907 builds\n" ] } ], "source": [ "import java.time.temporal.*\n", "import java.util.LinkedList\n", "\n", "val builds: List = runBlocking {\n", " api.buildsApi.getBuildsFlow(\n", " fromInstant = 0,\n", " query = \"\"\"buildStartTime>-7d\"\"\",\n", " models = listOf(BuildModelName.gradleAttributes),\n", " ).map {\n", " it.models!!.gradleAttributes!!.model!!\n", " }.toList(LinkedList())\n", "}\n", "\n", "println(\"${builds.size} builds\")\n", "check(builds.isNotEmpty()) { \"No builds found. Adjust query and try again.\" }" ] }, { "cell_type": "markdown", "id": "76f51226-513a-4bc3-8c6b-f4c5eaeffd6e", "metadata": {}, "source": [ "## Tables\n", "\n", "We'll now use [Kotlin/dataframe](https://github.com/Kotlin/dataframe) to visualize data" ] }, { "cell_type": "code", "execution_count": 3, "id": "5cd75a65-a819-497e-87c2-f0911cc1554f", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:24:47.969270Z", "start_time": "2024-04-01T20:24:45.457832Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ " " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%use dataframe(v=0.13.1)" ] }, { "cell_type": "markdown", "id": "18f6a5cb-408f-4086-93aa-84c357b886d6", "metadata": {}, "source": [ "Use `List.toDataFrame` to create a table of builds" ] }, { "cell_type": "code", "execution_count": 4, "id": "adbd028f-d30a-489e-94bd-d8f968a659a3", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:24:48.548709Z", "start_time": "2024-04-01T20:24:47.958241Z" }, "tags": [] }, "outputs": [ { "data": { "application/kotlindataframe+json": { "columns": [ "tasks", "count" ], "kotlin_dataframe": [ { "count": 4677, "tasks": "app:assembleBrazilDebug" }, { "count": 577, "tasks": "kotlinLSPProjectDeps" }, { "count": 420, "tasks": "IDE sync" }, { "count": 112, "tasks": "feature:home:impl:testReleaseUnitTest --tests br.com.ifood.home.*" }, { "count": 103, "tasks": "feature:hits:cards:impl:testReleaseUnitTest --tests br.com.ifood.hits.communitybuytaxonomyitemslist.data.CommunityBuyTaxonomyItemsCardResponseToUiMapperTest" }, { "count": 96, "tasks": "feature:splash:impl:testReleaseUnitTest --tests br.com.ifood.splash.animation.CommemorativeCustomizationDefaultServiceTest" }, { "count": 92, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.*" }, { "count": 90, "tasks": "feature:chat:core:impl:testReleaseUnitTest --tests br.com.ifood.chat.domain.service.ChatCallForegroundServiceTest" }, { "count": 73, "tasks": "feature:chat:core:impl:testReleaseUnitTest --tests br.com.ifood.chat.domain.service.ChatCallNotificationServiceTest" }, { "count": 70, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.presentation.plugin.standard.items.item.ReplacementsControllerTest.initReplacements_forceUiModelRemap" }, { "count": 58, "tasks": "feature:checkout:core:impl:testReleaseUnitTest" }, { "count": 54, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.domain.factory.DefaultCartRemoteConfigsFactoryTest" }, { "count": 54, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.domain.model.MerchantModelTest" }, { "count": 51, "tasks": "feature:chat:core:impl:testReleaseUnitTest --tests br.com.ifood.chat.domain.service.ChatCallForegroundServiceTest.onStartCommand_rejectCallIntent_stopsService" }, { "count": 48, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.presentation.plugin.standard.clubpurchase.ClubPurchasePluginViewModelTest" }, { "count": 48, "tasks": "feature:hits:page:impl:testReleaseUnitTest --tests br.com.ifood.hits.page.presentation.HitsPageViewModelTest" }, { "count": 45, "tasks": "feature:discoverycards:impl:testReleaseUnitTest" }, { "count": 44, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.domain.usecase.club.SetClubSummaryMetadataModelTest.invoke_withValidRulesAndRendering_returnCartWithUpdatedModel" }, { "count": 43, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.presentation.processor.viewmodel.OrderPoolingProcessorTest.resumeOrderPolling_withPaymentDataEventAndPaymentEventNotAuthorized_handlePaymentActionData" }, { "count": 41, "tasks": "feature:checkout:core:impl:testReleaseUnitTest --tests br.com.ifood.checkout.presentation.processor.viewmodel.ClubSubscriptionProcessorTest" } ], "ncol": 2, "nrow": 2901 }, "text/html": [ " \n", " \n", " \n", " \n", " \n", "
\n", "\n", "

... showing only top 20 of 2901 rows

DataFrame: rowsCount = 2901, columnsCount = 2

\n", "
taskscount
app:assembleBrazilDebug4677
kotlinLSPProjectDeps577
IDE sync420
feature:home:impl:testReleaseUnitTest...112
feature:hits:cards:impl:testReleaseUn...103
feature:splash:impl:testReleaseUnitTe...96
feature:checkout:core:impl:testReleas...92
feature:chat:core:impl:testReleaseUni...90
feature:chat:core:impl:testReleaseUni...73
feature:checkout:core:impl:testReleas...70
feature:checkout:core:impl:testReleas...58
feature:checkout:core:impl:testReleas...54
feature:checkout:core:impl:testReleas...54
feature:chat:core:impl:testReleaseUni...51
feature:checkout:core:impl:testReleas...48
feature:hits:page:impl:testReleaseUni...48
feature:discoverycards:impl:testRelea...45
feature:checkout:core:impl:testReleas...44
feature:checkout:core:impl:testReleas...43
feature:checkout:core:impl:testReleas...41
\n", " \n", " \n", " " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "val buildCounts = builds.toDataFrame {\n", " \"tasks\" from { build ->\n", " val tasks = build.requestedTasks.joinToString(\" \").trim(':')\n", " if (tasks.isNotBlank()) tasks\n", " else \"IDE sync\"\n", " }\n", "}.groupBy(\"tasks\").aggregate {\n", " count() into \"count\"\n", "}.sortByDesc(\"count\")\n", "\n", "// Jupyter will render the last cell line (a String, an Int, a DataFrame, etc.)\n", "buildCounts" ] }, { "cell_type": "markdown", "id": "7b1eb03b-eaae-44a6-b062-56bec70fb79d", "metadata": {}, "source": [ "## Plotting\n", "\n", "We'll use [Kotlin/kandy](https://github.com/Kotlin/kandy) for plotting the table. We'll only plot the top 5." ] }, { "cell_type": "code", "execution_count": 5, "id": "8c28e684-0f3b-43d9-a3d2-7b1ef91fa6c4", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:25:13.190618Z", "start_time": "2024-04-01T20:24:48.537811Z" } }, "outputs": [], "source": [ "%use kandy(v=0.6.0)" ] }, { "cell_type": "code", "execution_count": 13, "id": "293b2b4d", "metadata": { "ExecuteTime": { "end_time": "2024-04-01T20:25:13.226464Z", "start_time": "2024-04-01T20:24:50.790909Z" }, "tags": [] }, "outputs": [ { "data": { "application/plot+json": { "apply_color_scheme": true, "output": { "data": { "count": [ 4677, 577, 420 ], "tasks": [ "app:assembleBrazilDebug", "kotlinLSPProjectDeps", "IDE sync" ] }, "ggsize": { "height": 250, "width": 800 }, "kind": "plot", "layers": [ { "geom": "bar", "mapping": { "x": "count", "y": "tasks" }, "orientation": "y", "position": "dodge", "sampling": "none", "stat": "identity" } ], "mapping": {}, "scales": [ { "aesthetic": "x", "limits": [ null, null ] }, { "aesthetic": "y", "discrete": true } ] }, "output_type": "lets_plot_spec", "swing_enabled": true }, "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 0\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 500\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 1,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 1,500\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 2,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 2,500\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 3,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 3,500\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 4,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 4,500\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " app:assembleBrazilDebug\n", " \n", " \n", " \n", " \n", " \n", " \n", " kotlinLSPProjectDeps\n", " \n", " \n", " \n", " \n", " \n", " \n", " IDE sync\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " tasks\n", " \n", " \n", " \n", " \n", " count\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "plot(buildCounts.take(3)) {\n", " barsH {\n", " x(\"count\")\n", " y(\"tasks\")\n", " }\n", " layout.size = 800 to 250\n", "}" ] } ], "metadata": { "kernelspec": { "display_name": "Kotlin", "language": "kotlin", "name": "kotlin" }, "language_info": { "codemirror_mode": "text/x-kotlin", "file_extension": ".kt", "mimetype": "text/x-kotlin", "name": "kotlin", "nbconvert_exporter": "", "pygments_lexer": "kotlin", "version": "1.9.10" } }, "nbformat": 4, "nbformat_minor": 5 }